-- Logs begin at Tue 2022-07-12 21:23:56 UTC, end at Tue 2022-07-12 22:25:07 UTC. -- Jul 12 21:51:22.737209 dv-cinder-802955-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 12 21:51:22.744331 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 12 21:51:22.748174 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 12 21:51:22 2022] *** Jul 12 21:51:22.748174 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 12 21:51:22.748174 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 12 21:51:22.748174 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: nodename: dv-cinder-802955-7-iscsi Jul 12 21:51:22.748174 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: machine: x86_64 Jul 12 21:51:22.748174 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: clock source: unix Jul 12 21:51:22.748174 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: pcre jit disabled Jul 12 21:51:22.748174 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: detected number of CPU cores: 4 Jul 12 21:51:22.748174 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: current working directory: / Jul 12 21:51:22.748174 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: detected binary path: /usr/bin/uwsgi-core Jul 12 21:51:22.748174 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: your processes number limit is 47680 Jul 12 21:51:22.748174 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: your memory page size is 4096 bytes Jul 12 21:51:22.748174 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: detected max file descriptor number: 2048 Jul 12 21:51:22.748174 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: lock engine: pthread robust mutexes Jul 12 21:51:22.749083 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: thunder lock: enabled Jul 12 21:51:22.749083 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 12 21:51:22.749083 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 12 21:51:22.792291 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: Python main interpreter initialized at 0x5566d78559c0 Jul 12 21:51:22.792291 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: python threads support enabled Jul 12 21:51:22.792291 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: your server socket listen backlog is limited to 100 connections Jul 12 21:51:22.792291 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: your mercy for graceful operations on workers is 90 seconds Jul 12 21:51:22.792681 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: mapped 403077 bytes (393 KB) for 2 cores Jul 12 21:51:22.792790 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: *** Operational MODE: preforking *** Jul 12 21:51:22.793007 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: *** uWSGI is running in multiple interpreter mode *** Jul 12 21:51:22.793007 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: spawned uWSGI master process (pid: 80204) Jul 12 21:51:22.793220 dv-cinder-802955-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 12 21:51:22.794827 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: spawned uWSGI worker 1 (pid: 80218, cores: 1) Jul 12 21:51:22.794827 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: spawned uWSGI worker 2 (pid: 80219, cores: 1) Jul 12 21:51:22.794827 dv-cinder-802955-7-iscsi devstack@keystone.service[80204]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 12 21:51:28.294852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 21:51:28.294852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: from cryptography.utils import int_from_bytes Jul 12 21:51:28.295632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 21:51:28.295632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: from cryptography.utils import int_from_bytes Jul 12 21:51:28.298516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 21:51:28.298516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: from cryptography.utils import int_from_bytes Jul 12 21:51:28.298983 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 21:51:28.298983 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: from cryptography.utils import int_from_bytes Jul 12 21:51:28.344699 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 12 21:51:28.344903 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 12 21:51:28.345093 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] command line args: [] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 12 21:51:28.345365 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 12 21:51:28.345545 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ================================================================================ {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 12 21:51:28.345799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] admin_token = **** {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.346056 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] config_dir = [] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.346143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 12 21:51:28.346253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.346380 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 12 21:51:28.346472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] config_source = [] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.346549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] command line args: [] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 12 21:51:28.346631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] debug = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.346782 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.347037 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 12 21:51:28.347037 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ================================================================================ {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 12 21:51:28.347162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.347351 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] admin_token = **** {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.347437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.347564 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.347649 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] config_dir = [] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.347781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.347871 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.348025 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] list_limit = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.348122 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] config_source = [] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.348191 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] log_config_append = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.348250 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] debug = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.348449 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.348556 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.348643 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] log_dir = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.348702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] log_file = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.348784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.349007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.349115 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.349185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.349244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.349427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.349569 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.349678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.349824 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] list_limit = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.349914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.349985 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.350132 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] log_config_append = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.350233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.350311 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.350460 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.350550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] log_dir = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.350682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] log_file = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.350787 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.350912 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.351016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.351145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.351269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.351361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.351523 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.351611 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.351773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.351871 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.351951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.352098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.352193 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.352354 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.352589 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.352684 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.352790 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] publish_errors = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.352864 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.353025 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.353114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.353270 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.353359 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.353461 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.353556 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.353700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.353807 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.354543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.354627 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.354849 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] standard_threads = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.354934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.355076 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.355319 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.355430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.355529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.355613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] publish_errors = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.355692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] use_journal = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.355864 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] use_json = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.356045 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.356127 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] use_stderr = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.356269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.356343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] use_syslog = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.356418 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.356489 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.356558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.356627 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.357768 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.357860 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.357932 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] standard_threads = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.358001 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.358086 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.358170 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.358242 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.358312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.358381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.358381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] use_journal = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.358381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] use_json = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.358381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] use_stderr = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.358381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] use_syslog = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.358746 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 21:51:28.358746 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.358940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.359291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.359373 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.359446 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.359515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.359584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.359653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.359731 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.359901 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.359990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.360150 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.360240 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.360385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.360462 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.360633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.360736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.360899 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.360979 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.361146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.361229 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.361385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.361471 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.361620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.361698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.361873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.361956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.362117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.362191 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.362356 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.362441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.362594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.362670 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.362844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.362923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.363097 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.363208 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.363346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.363466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.363588 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.363720 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.363842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.363955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.364105 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.364183 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.364362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.364442 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.364614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.364699 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.364878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.364967 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.365115 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.365195 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.365346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.365423 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.365579 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.365665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.365828 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.365905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.366074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.366151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.366304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.366392 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.366538 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.366616 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.366771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.366849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.367011 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.367096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.367248 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.367324 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.367477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.367554 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.367709 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.367863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.367937 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.368006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.368157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.368235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.368387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.368481 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.368620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.368703 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.368880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.368960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.369114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.369201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.369343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.369545 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.369725 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.369804 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.370034 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.370156 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.370394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.370517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.370621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.370716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.370815 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.370918 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.371015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.371114 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.371228 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.371336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.371469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.371563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.371637 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.371755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.371871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.371994 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.372112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.372226 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.372352 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.372465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.372593 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.372724 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.372850 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.372956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.373096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.373208 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.373336 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.373448 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.373576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.373691 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.373814 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.373959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.374045 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.374141 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.374289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.374409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.374525 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.374655 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.374767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.374882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.375008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.375128 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.375279 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.375369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.375519 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.375599 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.375771 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.375854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.376019 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.376100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.376260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.376347 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.376501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.376588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.376761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.376846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.377002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.377092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.377238 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.377324 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.377475 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.377554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.377706 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.377784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.377940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.378019 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.378182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.378266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.378422 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.378502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.378661 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.378740 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.378896 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.378977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.379139 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.379219 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.379388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.379469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.379619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.379695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.379846 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.379926 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.380077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.380156 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.380309 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.380402 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.380540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.380621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.380800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.380894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.381034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.381113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.381267 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.381354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.381499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.381583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.381739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.381817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.381972 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.382051 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.382211 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.382297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.382450 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.382529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.382699 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.382778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.382930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] auth.external = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.383010 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.383162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.383303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.383438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.383518 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.383675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.383807 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.383938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] auth.external = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.384096 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] auth.password = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.384239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.384369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] auth.token = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.384369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.384621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.384800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.384939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.385098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.385236 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] auth.password = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.385363 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.385490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] auth.token = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.385617 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.385744 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.385878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.386013 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.386147 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.386282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.386406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] credential.caching = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.386535 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.386661 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.386807 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.386940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.387070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.387197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.387322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.387448 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.387581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] credential.caching = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.387710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.387837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.387963 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.388090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.388217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.388351 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.388434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.388571 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.388699 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.388843 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.388968 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.389104 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.389231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.389355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.389479 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.389602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.389726 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.389859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.389984 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.390117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.390252 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.390378 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.390503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] federation.caching = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.390634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.390759 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.390884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.391014 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.391140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.391266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.391357 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.391421 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.391533 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] federation.caching = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.391664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.391800 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.391925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.392057 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.392183 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.392308 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.392434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.392559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.392683 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.392837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.392967 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.393093 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.393218 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.393344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.393471 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] identity.caching = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.393614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.393743 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.393743 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.393983 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.394116 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.394245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.394449 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.394581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.394707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.394832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] identity.caching = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.394958 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.395091 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.395220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.395356 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.395483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.395609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.395756 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.395891 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.395970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.396042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.396113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.396193 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.396264 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.396347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.396420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.396491 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.396562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.396633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.396703 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.396880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.396955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.397026 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.397097 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.397174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.397246 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.397324 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.397398 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.397476 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.397548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.397626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.397699 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.397782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.397856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.397927 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.397997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.398067 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.398137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.398216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.398289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.398396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.398515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.398633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.398765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.398875 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.398997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.399114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.399242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.399356 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.399487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.399608 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.399722 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.399852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.399968 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.400105 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.400213 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.400340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.400443 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.400590 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.400670 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.401376 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.401531 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.401726 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.401812 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.401983 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.402084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.402235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.402315 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.402469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.402550 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.402712 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.402792 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.402948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.403028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.403190 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.403271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.403429 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.403509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.403673 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.403753 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.403914 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.403996 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.404155 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.404235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.404391 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.404470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.404630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.404729 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.404894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.404976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.405125 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.405204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.405364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.405446 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.405614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.405694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.405845 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.405924 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.406083 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.406162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.406323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.406410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.406558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.406645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.406792 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.406870 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.407022 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.407102 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.407259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.407347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.407494 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.407573 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.407731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.407810 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.407962 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.408049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.408201 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.408279 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.408433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.408512 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.408673 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.408819 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.408939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.409019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.409179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.409260 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.409406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.409494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.409654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.409734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.409890 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.409970 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.410127 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.410216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.410361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.410441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.410596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.410675 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.410826 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.410912 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.411073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.411152 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.411309 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.411387 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.411547 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.411635 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.411776 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.411855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.412009 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.412088 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.412244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.412331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.412480 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.412559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.412730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.412814 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.412989 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.413081 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.413218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.413298 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.413448 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.413527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.413684 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.413772 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.413918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.413996 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] resource.caching = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.414156 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.414235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.414398 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.414484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.414633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.414713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.414865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.414943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.415104 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.415192 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.415347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] resource.caching = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.415427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.415591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.415670 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.415837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.415924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.416073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.416152 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.416310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] role.caching = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.416420 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.417164 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.417323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] role.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.417528 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.417606 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.417678 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.417749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.417887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.417978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.418130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.418209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.418369 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] role.caching = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.418449 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.418620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] role.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.418709 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.418855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.418935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.419109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.419189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.419362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.419451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.419603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.419682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.419852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.419932 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.420101 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.420182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.420347 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.420427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.420593 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.420672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.420870 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.421020 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.421100 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.421256 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.421336 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.421505 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.421593 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.421755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.421835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.422002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.422082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.422253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.422436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.422517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.422676 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.422757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.422928 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.423015 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.423180 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.423261 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.423409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.423488 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.423642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.423725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.423876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.423958 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.424128 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.424208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.424382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.424470 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.424630 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.424774 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.424888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.424971 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.425121 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.425217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.425348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.425428 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.425585 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] token.caching = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.425665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.425825 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.425916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.426070 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.426151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.426310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.426389 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.426551 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.426640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.426785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.426866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.427018 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.427096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] token.caching = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.427252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.427339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.427494 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.427573 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.427731 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.427809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.427967 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.428056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.428205 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.428286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.428440 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.428520 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.428686 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.428793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.428958 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.429048 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.429193 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.429272 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.429434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.429520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.429680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.429766 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.429921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.430012 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.connection = **** {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.430169 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.430265 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.430418 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.430507 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.430650 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.430731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.430888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.430968 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.431127 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.431216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.431364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.431444 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.connection = **** {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.431599 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.431689 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.431834 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.431914 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.432104 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.432267 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.432365 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.432571 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.432651 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.432832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.432934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.433082 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.433163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.433318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.433398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.433561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.433649 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.433791 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.433871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.434034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.434114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.434269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.434359 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.434439 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80219) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 12 21:51:28.434542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.434798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.435038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.435286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.435521 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.435768 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.436008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 21:51:28.436182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80218) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 12 21:51:28.466770 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.466971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.467198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.469177 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.469585 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.469994 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.476791 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.481610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.508432 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.508657 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.511081 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.512141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.652622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.654172 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.702476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.702704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.702983 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.703221 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.767183 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.767416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.767641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.767729 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.767864 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.767951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.768081 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.768165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.768295 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.768378 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.768512 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.768748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.793836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.794032 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.794120 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.794257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.794345 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.794482 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.794567 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.794701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.794795 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.794921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.795064 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.795148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.795289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.795459 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.808757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.809259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.822892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.823090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.823329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.823442 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.823648 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.823745 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.823822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.823894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.823984 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.824061 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.824179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.824262 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.824390 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80219) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.824473 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80218) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 21:51:28.842025 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.842231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 21:51:28.842316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.842495 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 21:51:28.842578 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.842752 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 21:51:28.842836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.843021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.843104 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 21:51:28.843282 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.843375 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.843538 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.843631 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.843788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.843879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.844065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.855073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.855343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.855428 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.855609 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.855694 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.855893 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.855987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.856166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.864085 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.864412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.867370 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.867654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.867740 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.867935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.868021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.868216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.868310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.868505 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.868612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.868780 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.868868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.869053 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.869137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.869324 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.869406 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.870149 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.870284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.870417 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.870557 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.870692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.870818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.870945 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.897019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.897418 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.898642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.898921 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.899062 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.899344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.904824 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.905056 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.905253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.905457 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.905542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.905806 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.912683 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.912997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.913229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.913516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.917531 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.917816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.918095 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.918179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.918379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.918479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.918659 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.918743 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.918933 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.919017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.919202 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.919295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.919470 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.919554 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.919749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.920021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.938456 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.938727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.939110 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.939391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.943567 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.944157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.947187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.947482 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.948165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.948496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.952672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.953662 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.956271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.957306 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.959951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.960249 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.960694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.961013 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.961179 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.961319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.961966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.962077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.962188 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.962289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.962388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.962498 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.984310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.984603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.984890 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.985151 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.987018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.987315 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.987584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.987849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.993016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.994748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.995159 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.995791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.997523 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.997798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.998350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:28.998626 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.998903 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.999185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:28.999467 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.000996 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:29.001280 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.001563 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.001847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.002137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.012849 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:29.013129 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.013407 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.013686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.013966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.015573 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:29.015670 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.015860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.015972 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.016142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.016448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.016751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.017044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.017321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.034326 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:29.037607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:29.038221 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:29.040891 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.042093 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:29.043296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.044933 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:29.045212 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.045413 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.045499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.047027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.048814 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:29.049099 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.049389 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.052730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.054314 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:29.056645 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.057597 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.057880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.058276 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.058554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.058771 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:29.063929 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.064227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.064549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.065067 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.067844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:29.068129 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.068403 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.075329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:29.075623 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.075904 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.076345 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80219) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:29.076626 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.076929 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.077206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.077475 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.077860 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.078138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.078424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.078705 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.078975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.079245 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.079506 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.079769 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.080038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80219) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.083771 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80218) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 21:51:29.084045 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.084326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.084603 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.084907 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.085185 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.085464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.085744 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.086136 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.086411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.086693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.086959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.087228 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.087499 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80218) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 21:51:29.118604 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5566d78559c0 pid: 80219 (default app) Jul 12 21:51:29.122217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fec9f-1df5-4e37-8454-57ead7619d22 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:29.122502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fec9f-1df5-4e37-8454-57ead7619d22 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:29.122778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fec9f-1df5-4e37-8454-57ead7619d22 None None] PATH_INFO: `/v3` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:29.126082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5566d78559c0 pid: 80218 (default app) Jul 12 21:51:29.127041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1/1] 10.222.0.44 () {50 vars in 791 bytes} [Tue Jul 12 21:51:29 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:37.023023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f97bd1-32ea-4031-816f-bf74a53225c4 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:37.023598 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f97bd1-32ea-4031-816f-bf74a53225c4 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:37.023598 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f97bd1-32ea-4031-816f-bf74a53225c4 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:37.028066 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1/2] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:51:37.033390 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f488a-5287-4880-9075-fc0a3ee2c0c2 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:37.033836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f488a-5287-4880-9075-fc0a3ee2c0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:37.033928 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f488a-5287-4880-9075-fc0a3ee2c0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:37.102491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: INFO dbcounter [None req-ea4f488a-5287-4880-9075-fc0a3ee2c0c2 None None] Registered counter for database keystone Jul 12 21:51:37.125584 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG oslo_db.sqlalchemy.engines [None req-ea4f488a-5287-4880-9075-fc0a3ee2c0c2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80219) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 12 21:51:37.131903 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writer thread running {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 12 21:51:37.310315 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG passlib.handlers.bcrypt [None req-ea4f488a-5287-4880-9075-fc0a3ee2c0c2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80219) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 12 21:51:37.310734 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG passlib.handlers.bcrypt [None req-ea4f488a-5287-4880-9075-fc0a3ee2c0c2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80219) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 12 21:51:37.364937 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-ea4f488a-5287-4880-9075-fc0a3ee2c0c2 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:37.366538 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ea4f488a-5287-4880-9075-fc0a3ee2c0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:37.488675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 2/3] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 12 21:51:37.561213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f97bd1-32ea-4031-816f-bf74a53225c4 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:37.631309 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: INFO dbcounter [None req-e5f97bd1-32ea-4031-816f-bf74a53225c4 None None] Registered counter for database keystone Jul 12 21:51:37.651491 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG oslo_db.sqlalchemy.engines [None req-e5f97bd1-32ea-4031-816f-bf74a53225c4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80218) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 12 21:51:37.657954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writer thread running {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 12 21:51:37.779163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f97bd1-32ea-4031-816f-bf74a53225c4 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:37.784306 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42497d-0e0e-47d3-827f-f7e4507c0412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:37.785015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42497d-0e0e-47d3-827f-f7e4507c0412 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:37.785306 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42497d-0e0e-47d3-827f-f7e4507c0412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:37.785588 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42497d-0e0e-47d3-827f-f7e4507c0412 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:37.795037 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42497d-0e0e-47d3-827f-f7e4507c0412 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:37.851626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6d42497d-0e0e-47d3-827f-f7e4507c0412 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 12 21:51:37.851894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42497d-0e0e-47d3-827f-f7e4507c0412 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:37.862794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 2/4] 10.222.0.44 () {58 vars in 1324 bytes} [Tue Jul 12 21:51:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 21:51:37.868869 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f488a-5287-4880-9075-fc0a3ee2c0c2 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:37.884346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f488a-5287-4880-9075-fc0a3ee2c0c2 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:37.889459 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392bf0c-3a1e-4e0c-be71-7e986871ff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:37.890605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392bf0c-3a1e-4e0c-be71-7e986871ff8f admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:37.890896 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392bf0c-3a1e-4e0c-be71-7e986871ff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:37.891178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392bf0c-3a1e-4e0c-be71-7e986871ff8f admin admin] PATH_INFO: `/v3/projects/3e825823085549c0909b324f2e7acb2a` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:37.892690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392bf0c-3a1e-4e0c-be71-7e986871ff8f admin admin] RBAC: Authorizing `identity:get_project(project_id=3e825823085549c0909b324f2e7acb2a)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:37.946286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a392bf0c-3a1e-4e0c-be71-7e986871ff8f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 12 21:51:37.946637 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392bf0c-3a1e-4e0c-be71-7e986871ff8f admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:37.951429 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 3/5] 10.222.0.44 () {56 vars in 1207 bytes} [Tue Jul 12 21:51:37 2022] GET /identity/v3/projects/3e825823085549c0909b324f2e7acb2a => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:39.991515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdca6b-830d-4786-8675-e46ce4bc4ff9 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:39.991959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdca6b-830d-4786-8675-e46ce4bc4ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:39.992060 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdca6b-830d-4786-8675-e46ce4bc4ff9 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:39.996205 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 3/6] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:51:40.001646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc474443-62a1-4302-80db-1275f352cb9e None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:40.002074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc474443-62a1-4302-80db-1275f352cb9e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:40.002280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc474443-62a1-4302-80db-1275f352cb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:40.021383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-fc474443-62a1-4302-80db-1275f352cb9e None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:40.022926 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-fc474443-62a1-4302-80db-1275f352cb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:40.031478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 4/7] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 12 21:51:40.053849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afdca6b-830d-4786-8675-e46ce4bc4ff9 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:40.067314 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afdca6b-830d-4786-8675-e46ce4bc4ff9 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:40.071599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05afd38e-832a-453b-917d-aa8385fe6811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:40.072251 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afd38e-832a-453b-917d-aa8385fe6811 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:40.072588 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afd38e-832a-453b-917d-aa8385fe6811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:40.072957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afd38e-832a-453b-917d-aa8385fe6811 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:40.081700 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05afd38e-832a-453b-917d-aa8385fe6811 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:40.083519 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05afd38e-832a-453b-917d-aa8385fe6811 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:40.093610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 4/8] 10.222.0.44 () {58 vars in 1324 bytes} [Tue Jul 12 21:51:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 21:51:40.099469 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc474443-62a1-4302-80db-1275f352cb9e None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:40.109681 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc474443-62a1-4302-80db-1275f352cb9e None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:40.114122 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334dc87-5cd0-40e8-8621-d084a67b0a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:40.115021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334dc87-5cd0-40e8-8621-d084a67b0a78 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:40.115307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334dc87-5cd0-40e8-8621-d084a67b0a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:40.115586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334dc87-5cd0-40e8-8621-d084a67b0a78 admin admin] PATH_INFO: `/v3/users/a08b779b7e9045028b1b3a86939d306f` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:40.117077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3334dc87-5cd0-40e8-8621-d084a67b0a78 admin admin] RBAC: Authorizing `identity:get_user(user_id=a08b779b7e9045028b1b3a86939d306f)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:40.118971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3334dc87-5cd0-40e8-8621-d084a67b0a78 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:40.121377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 5/9] 10.222.0.44 () {56 vars in 1198 bytes} [Tue Jul 12 21:51:40 2022] GET /identity/v3/users/a08b779b7e9045028b1b3a86939d306f => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:42.257351 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359dfbf-76a2-463f-950e-406ce2f3120d None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:42.257844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359dfbf-76a2-463f-950e-406ce2f3120d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:42.257844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359dfbf-76a2-463f-950e-406ce2f3120d None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:42.259186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 5/10] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:51:42.264750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f717c-a0e5-4ad9-a675-628094f7c41c None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:42.265203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f717c-a0e5-4ad9-a675-628094f7c41c None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:42.265327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f717c-a0e5-4ad9-a675-628094f7c41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:42.282556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-689f717c-a0e5-4ad9-a675-628094f7c41c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:42.283750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-689f717c-a0e5-4ad9-a675-628094f7c41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:42.302361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b67a9a-9fd1-4394-82ff-302b0b8f506b None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:42.302891 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b67a9a-9fd1-4394-82ff-302b0b8f506b None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:42.303177 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b67a9a-9fd1-4394-82ff-302b0b8f506b None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:42.304531 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 6/11] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:51:42.310559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093e0f3-53ae-40d1-ac5d-f65c28accbb7 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:42.310852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093e0f3-53ae-40d1-ac5d-f65c28accbb7 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:42.311136 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093e0f3-53ae-40d1-ac5d-f65c28accbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:42.350640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG passlib.handlers.bcrypt [None req-7093e0f3-53ae-40d1-ac5d-f65c28accbb7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80218) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 12 21:51:42.351063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG passlib.handlers.bcrypt [None req-7093e0f3-53ae-40d1-ac5d-f65c28accbb7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80218) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 12 21:51:42.367777 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 6/12] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:51:42.395529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f717c-a0e5-4ad9-a675-628094f7c41c None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:42.397312 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-7093e0f3-53ae-40d1-ac5d-f65c28accbb7 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:42.398902 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-7093e0f3-53ae-40d1-ac5d-f65c28accbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:42.420846 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:42.500625 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 7/13] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:51:42.505111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f717c-a0e5-4ad9-a675-628094f7c41c None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:42.523815 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093e0f3-53ae-40d1-ac5d-f65c28accbb7 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:42.538573 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55022a-a5e9-489b-bbe8-008845302356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:42.539232 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55022a-a5e9-489b-bbe8-008845302356 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:42.539516 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55022a-a5e9-489b-bbe8-008845302356 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:42.539797 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55022a-a5e9-489b-bbe8-008845302356 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:42.610331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c55022a-a5e9-489b-bbe8-008845302356 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:42.611828 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c55022a-a5e9-489b-bbe8-008845302356 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:42.630483 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093e0f3-53ae-40d1-ac5d-f65c28accbb7 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:42.632823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:42.635494 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:42.671020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3a406-7804-43d1-9fb8-fb7c62f4bdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:42.672073 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3a406-7804-43d1-9fb8-fb7c62f4bdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:42.672367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3a406-7804-43d1-9fb8-fb7c62f4bdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:42.672638 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3a406-7804-43d1-9fb8-fb7c62f4bdd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:42.685445 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3a406-7804-43d1-9fb8-fb7c62f4bdd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:51:42.697044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-2aa3a406-7804-43d1-9fb8-fb7c62f4bdd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:51:42.697044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:51:42.697044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:51:42.697044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:51:42.697044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:51:42.697044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:51:42.697044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:51:42.697044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:51:42.697044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:51:42.697044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:51:42.697044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:51:42.697044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:51:42.697044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:51:42.697044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:51:42.697044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:51:42.697044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:51:42.698586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:51:42.698586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:51:42.698586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:51:42.698586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:51:42.698586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:51:42.698586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:51:42.698586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:51:42.698586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:51:42.698586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:51:42.698586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:51:42.698586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:51:42.698586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:51:42.698586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:51:42.698586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:51:42.698586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:51:42.698586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:51:42.699622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:51:42.699622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:51:42.699622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:51:42.699622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:51:42.699622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:51:42.699622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:51:42.699622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:51:42.699622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:51:42.699622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:51:42.699622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:51:42.699622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:51:42.699622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:51:42.699622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 8/14] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:51:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:51:42.704704 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3a406-7804-43d1-9fb8-fb7c62f4bdd3 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:42.716239 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 7/15] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:51:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 21:51:42.721713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55022a-a5e9-489b-bbe8-008845302356 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:42.804583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3a406-7804-43d1-9fb8-fb7c62f4bdd3 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:42.821454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55022a-a5e9-489b-bbe8-008845302356 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:42.840298 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb20d34-a0db-4e54-ac6f-7515b44065d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:42.840992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb20d34-a0db-4e54-ac6f-7515b44065d9 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:42.841282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb20d34-a0db-4e54-ac6f-7515b44065d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:42.841564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb20d34-a0db-4e54-ac6f-7515b44065d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:42.842329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb20d34-a0db-4e54-ac6f-7515b44065d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:51:42.842641 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb20d34-a0db-4e54-ac6f-7515b44065d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:42.844101 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb20d34-a0db-4e54-ac6f-7515b44065d9 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:42.845815 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:42.856480 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 9/16] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:51:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:51:42.857890 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2623f0-e35f-46c3-bc42-db0229e17a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:42.858698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2623f0-e35f-46c3-bc42-db0229e17a92 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:42.858979 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2623f0-e35f-46c3-bc42-db0229e17a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:42.859693 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2623f0-e35f-46c3-bc42-db0229e17a92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:42.861978 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb20d34-a0db-4e54-ac6f-7515b44065d9 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:42.870377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2623f0-e35f-46c3-bc42-db0229e17a92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:42.871869 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2623f0-e35f-46c3-bc42-db0229e17a92 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:42.889326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:42.899044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-bc2623f0-e35f-46c3-bc42-db0229e17a92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:51:42.899044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:51:42.899044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:51:42.899044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:51:42.899044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:51:42.899044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:51:42.899044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:51:42.899044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:51:42.899044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:51:42.899044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:51:42.899044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:51:42.899044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:51:42.899044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:51:42.899044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:51:42.899044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:51:42.900045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:51:42.900045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:51:42.900045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:51:42.900045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:51:42.900045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:51:42.900045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:51:42.900045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:51:42.900045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:51:42.900045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:51:42.900045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:51:42.900045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:51:42.900045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:51:42.900045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:51:42.900045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:51:42.900045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:51:42.900045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:51:42.900045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:51:42.901189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:51:42.901189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:51:42.901189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:51:42.901189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:51:42.901189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 8/17] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:51:42.907425 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2623f0-e35f-46c3-bc42-db0229e17a92 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:42.967385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb20d34-a0db-4e54-ac6f-7515b44065d9 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:43.002513 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3180a9e-1545-41ce-8ee0-4e24e3f8f935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:43.003391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3180a9e-1545-41ce-8ee0-4e24e3f8f935 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:43.003681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3180a9e-1545-41ce-8ee0-4e24e3f8f935 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:43.003961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3180a9e-1545-41ce-8ee0-4e24e3f8f935 None admin] PATH_INFO: `/v3/users/a08b779b7e9045028b1b3a86939d306f` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:43.005804 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3180a9e-1545-41ce-8ee0-4e24e3f8f935 None admin] RBAC: Authorizing `identity:get_user(user_id=a08b779b7e9045028b1b3a86939d306f)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:43.007282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3180a9e-1545-41ce-8ee0-4e24e3f8f935 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:43.008958 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 10/18] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:51:42 2022] GET /identity/v3/users/a08b779b7e9045028b1b3a86939d306f => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:51:43.009629 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2623f0-e35f-46c3-bc42-db0229e17a92 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:43.015451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3180a9e-1545-41ce-8ee0-4e24e3f8f935 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:43.045108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722a28e-cbad-4ad9-8501-317586aa7c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:43.045779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722a28e-cbad-4ad9-8501-317586aa7c6e None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:43.046071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722a28e-cbad-4ad9-8501-317586aa7c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:43.046355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722a28e-cbad-4ad9-8501-317586aa7c6e None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:43.047090 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722a28e-cbad-4ad9-8501-317586aa7c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:51:43.047402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722a28e-cbad-4ad9-8501-317586aa7c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:43.048825 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722a28e-cbad-4ad9-8501-317586aa7c6e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:43.067000 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 9/19] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:51:43.118457 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:43.126921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3180a9e-1545-41ce-8ee0-4e24e3f8f935 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:43.163121 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22837081-8392-4369-acd3-e8c1f4501670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:43.163938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22837081-8392-4369-acd3-e8c1f4501670 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:43.164228 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22837081-8392-4369-acd3-e8c1f4501670 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:43.164510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22837081-8392-4369-acd3-e8c1f4501670 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:43.165786 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22837081-8392-4369-acd3-e8c1f4501670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:43.167366 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22837081-8392-4369-acd3-e8c1f4501670 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:43.169096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 11/20] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:51:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:51:43.175354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722a28e-cbad-4ad9-8501-317586aa7c6e None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:43.213439 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:43.283761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722a28e-cbad-4ad9-8501-317586aa7c6e None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:43.321754 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b083d-82f5-4cb2-a85b-0e3c508b3aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:43.323331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b083d-82f5-4cb2-a85b-0e3c508b3aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:43.323599 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b083d-82f5-4cb2-a85b-0e3c508b3aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:43.323876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b083d-82f5-4cb2-a85b-0e3c508b3aa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:43.324777 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b083d-82f5-4cb2-a85b-0e3c508b3aa0 None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.domain.id=default, user.id=a08b779b7e9045028b1b3a86939d306f) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:51:43.325067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b083d-82f5-4cb2-a85b-0e3c508b3aa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:43.326557 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b083d-82f5-4cb2-a85b-0e3c508b3aa0 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:43.335905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 10/21] 10.222.0.44 () {56 vars in 1322 bytes} [Tue Jul 12 21:51:43 2022] GET /identity/v3/role_assignments?user.id=a08b779b7e9045028b1b3a86939d306f&scope.domain.id=default&role.id=053d1730ed764973abb1490894e07681 => generated 233 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:51:45.492764 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d868132-86f3-4bb3-b008-179185f854ac None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:45.493786 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d868132-86f3-4bb3-b008-179185f854ac None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:45.494323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d868132-86f3-4bb3-b008-179185f854ac None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:45.496254 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 12/22] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:51:45.501450 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a595851-26c5-4ffe-b014-13363405c519 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:45.502312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a595851-26c5-4ffe-b014-13363405c519 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:45.502795 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a595851-26c5-4ffe-b014-13363405c519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:45.526396 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-0a595851-26c5-4ffe-b014-13363405c519 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:45.528013 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0a595851-26c5-4ffe-b014-13363405c519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:45.552432 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8371e-ad59-46be-9a74-ecc2c551f2d0 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:45.553082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8371e-ad59-46be-9a74-ecc2c551f2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:45.553571 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8371e-ad59-46be-9a74-ecc2c551f2d0 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:45.555465 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 13/23] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:51:45.562029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b7780-cabf-4ca6-8775-7b8116df3fb6 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:45.562314 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b7780-cabf-4ca6-8775-7b8116df3fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:45.562591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b7780-cabf-4ca6-8775-7b8116df3fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:45.595994 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-427b7780-cabf-4ca6-8775-7b8116df3fb6 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:45.597312 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-427b7780-cabf-4ca6-8775-7b8116df3fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:45.677523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 11/24] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:51:45.722323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a595851-26c5-4ffe-b014-13363405c519 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:45.743263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 14/25] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:51:45.789774 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:45.790385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b7780-cabf-4ca6-8775-7b8116df3fb6 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:45.974114 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a595851-26c5-4ffe-b014-13363405c519 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:46.031601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643e831-23d7-471f-a558-2c0ea2e851fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:46.032412 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643e831-23d7-471f-a558-2c0ea2e851fe None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:46.032693 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643e831-23d7-471f-a558-2c0ea2e851fe None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:46.041314 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643e831-23d7-471f-a558-2c0ea2e851fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:46.058581 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643e831-23d7-471f-a558-2c0ea2e851fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:51:46.063177 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:46.068073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-8643e831-23d7-471f-a558-2c0ea2e851fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:51:46.068073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:51:46.068073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:51:46.068073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:51:46.068073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:51:46.068073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:51:46.068073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:51:46.068073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:51:46.068073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:51:46.068073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:51:46.068073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:51:46.068073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:51:46.068073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:51:46.068073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:51:46.068073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:51:46.068073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:51:46.068971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:51:46.068971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:51:46.068971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:51:46.068971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:51:46.068971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:51:46.068971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:51:46.068971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:51:46.068971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:51:46.068971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:51:46.068971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:51:46.068971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:51:46.068971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:51:46.068971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:51:46.068971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:51:46.068971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:51:46.068971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:51:46.069721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:51:46.069721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:51:46.069721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:51:46.069721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:51:46.069721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:51:46.069721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:51:46.069721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:51:46.069721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:51:46.069721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:51:46.069721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:51:46.069721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:51:46.069721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:51:46.075257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b7780-cabf-4ca6-8775-7b8116df3fb6 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:46.077484 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 12/26] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:51:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:51:46.082531 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03458f58-2d8d-455d-bcc2-8b5182120d5f None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:46.082819 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03458f58-2d8d-455d-bcc2-8b5182120d5f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:46.083097 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03458f58-2d8d-455d-bcc2-8b5182120d5f None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:46.084441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 13/27] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:51:46.086926 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03458f58-2d8d-455d-bcc2-8b5182120d5f None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:46.133457 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af058fbc-a06f-44a2-b941-0707fbb90ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:46.134278 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af058fbc-a06f-44a2-b941-0707fbb90ced None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:46.134562 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af058fbc-a06f-44a2-b941-0707fbb90ced None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:46.135434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af058fbc-a06f-44a2-b941-0707fbb90ced None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:46.175074 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af058fbc-a06f-44a2-b941-0707fbb90ced None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:46.176771 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af058fbc-a06f-44a2-b941-0707fbb90ced None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:46.197100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:46.214230 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03458f58-2d8d-455d-bcc2-8b5182120d5f None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:46.222118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-af058fbc-a06f-44a2-b941-0707fbb90ced None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:51:46.222118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:51:46.222118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:51:46.222118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:51:46.222118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:51:46.222118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:51:46.222118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:51:46.222118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:51:46.222118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:51:46.222118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:51:46.222118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:51:46.222118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:51:46.222118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:51:46.222118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:51:46.222118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:51:46.223079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:51:46.223079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:51:46.223079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:51:46.223079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:51:46.223079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:51:46.223079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:51:46.223079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:51:46.223079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:51:46.223079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:51:46.223079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:51:46.223079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:51:46.223079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:51:46.223079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:51:46.223079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:51:46.223079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:51:46.223079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:51:46.223079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:51:46.224058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:51:46.224058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:51:46.224058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:51:46.224058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:51:46.225668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 15/28] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:51:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 436 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:51:46.232732 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868d548-46f7-4a3e-aaa3-c801d6c15e3e None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:46.233315 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868d548-46f7-4a3e-aaa3-c801d6c15e3e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:46.233763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868d548-46f7-4a3e-aaa3-c801d6c15e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:46.256468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b615a1-b3e5-47cb-94fe-3af5145924b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:46.257582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b615a1-b3e5-47cb-94fe-3af5145924b5 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:46.258156 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b615a1-b3e5-47cb-94fe-3af5145924b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:46.258661 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b615a1-b3e5-47cb-94fe-3af5145924b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:46.259745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b615a1-b3e5-47cb-94fe-3af5145924b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:51:46.260585 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b615a1-b3e5-47cb-94fe-3af5145924b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:46.262678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b615a1-b3e5-47cb-94fe-3af5145924b5 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:46.273804 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-6868d548-46f7-4a3e-aaa3-c801d6c15e3e None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:46.275071 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-6868d548-46f7-4a3e-aaa3-c801d6c15e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:46.279061 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 14/29] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:51:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:51:46.280182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b615a1-b3e5-47cb-94fe-3af5145924b5 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:46.387594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 16/30] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:51:46.389144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868d548-46f7-4a3e-aaa3-c801d6c15e3e None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:46.398553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b615a1-b3e5-47cb-94fe-3af5145924b5 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:46.451124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b7b05-b8ae-46b1-b54e-630631ad2457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:46.452268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b7b05-b8ae-46b1-b54e-630631ad2457 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:46.453015 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b7b05-b8ae-46b1-b54e-630631ad2457 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:46.453499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b7b05-b8ae-46b1-b54e-630631ad2457 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:46.454433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b7b05-b8ae-46b1-b54e-630631ad2457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:51:46.454923 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b7b05-b8ae-46b1-b54e-630631ad2457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:46.457287 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b7b05-b8ae-46b1-b54e-630631ad2457 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:46.473345 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 15/31] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:46.475754 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90838a4-d31b-4cdd-aa94-730e5f2785e1 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:46.476232 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90838a4-d31b-4cdd-aa94-730e5f2785e1 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:46.476744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90838a4-d31b-4cdd-aa94-730e5f2785e1 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:46.479224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 16/32] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:51:46.480255 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90838a4-d31b-4cdd-aa94-730e5f2785e1 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:46.484149 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:46.497369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:46.530266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868d548-46f7-4a3e-aaa3-c801d6c15e3e None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:46.576093 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14850938-bdc1-4628-b79a-d563e84b64e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:46.577778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14850938-bdc1-4628-b79a-d563e84b64e2 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:46.578245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14850938-bdc1-4628-b79a-d563e84b64e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:46.578820 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14850938-bdc1-4628-b79a-d563e84b64e2 None admin] PATH_INFO: `/v3/users/a08b779b7e9045028b1b3a86939d306f` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:46.580346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14850938-bdc1-4628-b79a-d563e84b64e2 None admin] RBAC: Authorizing `identity:get_user(user_id=a08b779b7e9045028b1b3a86939d306f)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:46.582069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14850938-bdc1-4628-b79a-d563e84b64e2 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:46.589554 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 17/33] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:51:46 2022] GET /identity/v3/users/a08b779b7e9045028b1b3a86939d306f => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:46.590158 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14850938-bdc1-4628-b79a-d563e84b64e2 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:46.636676 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90838a4-d31b-4cdd-aa94-730e5f2785e1 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:46.681942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c075aba-8b74-4afc-862b-c698f7eef36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:46.682540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c075aba-8b74-4afc-862b-c698f7eef36b None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:46.682822 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c075aba-8b74-4afc-862b-c698f7eef36b None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:46.683098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c075aba-8b74-4afc-862b-c698f7eef36b None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:46.683860 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c075aba-8b74-4afc-862b-c698f7eef36b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:46.685394 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c075aba-8b74-4afc-862b-c698f7eef36b None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:46.717001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14850938-bdc1-4628-b79a-d563e84b64e2 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:46.753591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65d9d3-d9b0-453c-830c-a19af8c4a78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:46.754607 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65d9d3-d9b0-453c-830c-a19af8c4a78f None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:46.755112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65d9d3-d9b0-453c-830c-a19af8c4a78f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:46.755624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65d9d3-d9b0-453c-830c-a19af8c4a78f None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:46.756541 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65d9d3-d9b0-453c-830c-a19af8c4a78f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:46.758360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65d9d3-d9b0-453c-830c-a19af8c4a78f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:46.774066 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:46.909604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 18/34] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Jul 12 21:51:46 2022] POST /identity/v3/projects => generated 301 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:51:46.920989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199880a-d719-40c0-9fa7-6b804461eb09 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:46.920989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199880a-d719-40c0-9fa7-6b804461eb09 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:46.920989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199880a-d719-40c0-9fa7-6b804461eb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:46.929132 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 17/35] 10.222.0.44 () {60 vars in 1131 bytes} [Tue Jul 12 21:51:46 2022] POST /identity/v3/roles => generated 219 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:51:46.930170 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c075aba-8b74-4afc-862b-c698f7eef36b None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:46.962556 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-2199880a-d719-40c0-9fa7-6b804461eb09 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:46.966874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2199880a-d719-40c0-9fa7-6b804461eb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:47.017396 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:47.060077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c075aba-8b74-4afc-862b-c698f7eef36b None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:47.127197 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 19/36] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:51:47.128575 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59878cfe-1dae-48f4-8418-96c1ce8d6f11 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:47.129066 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59878cfe-1dae-48f4-8418-96c1ce8d6f11 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:47.129535 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59878cfe-1dae-48f4-8418-96c1ce8d6f11 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:47.130859 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0ee18-9aef-4910-8be5-0d6cf83bdf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:47.131314 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 20/37] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:51:47.135640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0ee18-9aef-4910-8be5-0d6cf83bdf04 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:47.135640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0ee18-9aef-4910-8be5-0d6cf83bdf04 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:47.135640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0ee18-9aef-4910-8be5-0d6cf83bdf04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:47.135640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a0ee18-9aef-4910-8be5-0d6cf83bdf04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:47.136743 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a0ee18-9aef-4910-8be5-0d6cf83bdf04 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:47.137768 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cef15d-5e98-4969-a573-b3aaccc82a52 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:47.138166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cef15d-5e98-4969-a573-b3aaccc82a52 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:47.138547 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cef15d-5e98-4969-a573-b3aaccc82a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:47.139292 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 18/38] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:51:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:47.145937 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0ee18-9aef-4910-8be5-0d6cf83bdf04 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:47.168926 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-b6cef15d-5e98-4969-a573-b3aaccc82a52 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:47.170515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b6cef15d-5e98-4969-a573-b3aaccc82a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:47.271686 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 21/39] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:51:47.272786 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cef15d-5e98-4969-a573-b3aaccc82a52 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:47.281701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0ee18-9aef-4910-8be5-0d6cf83bdf04 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:47.319531 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b862e34-19c4-4f99-adaf-6b983c265f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:47.320416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b862e34-19c4-4f99-adaf-6b983c265f65 None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:47.320699 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b862e34-19c4-4f99-adaf-6b983c265f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:47.321006 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b862e34-19c4-4f99-adaf-6b983c265f65 None admin] PATH_INFO: `/v3/domains/default/users/a08b779b7e9045028b1b3a86939d306f/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:47.323826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b862e34-19c4-4f99-adaf-6b983c265f65 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a08b779b7e9045028b1b3a86939d306f, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:47.325344 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b862e34-19c4-4f99-adaf-6b983c265f65 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:47.334785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:47.335527 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:47.341735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 19/40] 10.222.0.44 () {58 vars in 1361 bytes} [Tue Jul 12 21:51:47 2022] PUT /identity/v3/domains/default/users/a08b779b7e9045028b1b3a86939d306f/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:51:47.356759 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24a7e4-d1a3-4c78-a03e-191a36dd1bad None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:47.357378 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24a7e4-d1a3-4c78-a03e-191a36dd1bad None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:47.357920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24a7e4-d1a3-4c78-a03e-191a36dd1bad None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:47.359684 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 20/41] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:47 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:51:47.360652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24a7e4-d1a3-4c78-a03e-191a36dd1bad None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:47.392532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cef15d-5e98-4969-a573-b3aaccc82a52 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:47.437436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d980ef-8071-4273-a89c-a8c678055ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:47.438578 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d980ef-8071-4273-a89c-a8c678055ddf None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:47.439169 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d980ef-8071-4273-a89c-a8c678055ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:47.439458 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d980ef-8071-4273-a89c-a8c678055ddf None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:47.440247 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d980ef-8071-4273-a89c-a8c678055ddf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:47.442140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d980ef-8071-4273-a89c-a8c678055ddf None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:47.460581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 22/42] 10.222.0.44 () {60 vars in 1131 bytes} [Tue Jul 12 21:51:47 2022] POST /identity/v3/roles => generated 225 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:51:47.462039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8eccd-e70c-4945-bfaa-c1cc37f9eb38 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:47.462339 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8eccd-e70c-4945-bfaa-c1cc37f9eb38 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:47.462628 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8eccd-e70c-4945-bfaa-c1cc37f9eb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:47.485396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-e9b8eccd-e70c-4945-bfaa-c1cc37f9eb38 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:47.486824 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e9b8eccd-e70c-4945-bfaa-c1cc37f9eb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:47.510624 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24a7e4-d1a3-4c78-a03e-191a36dd1bad None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:47.561515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d60cea-4248-4948-b7ca-4ba6cdecef39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:47.562542 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d60cea-4248-4948-b7ca-4ba6cdecef39 None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:47.563062 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d60cea-4248-4948-b7ca-4ba6cdecef39 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:47.563607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d60cea-4248-4948-b7ca-4ba6cdecef39 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:47.564608 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d60cea-4248-4948-b7ca-4ba6cdecef39 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:47.566551 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d60cea-4248-4948-b7ca-4ba6cdecef39 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:47.582026 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 23/43] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:51:47.588866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 21/44] 10.222.0.44 () {60 vars in 1131 bytes} [Tue Jul 12 21:51:47 2022] POST /identity/v3/roles => generated 223 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:51:47.608782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8eccd-e70c-4945-bfaa-c1cc37f9eb38 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:47.670576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:47.743848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8eccd-e70c-4945-bfaa-c1cc37f9eb38 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:47.786354 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01184dd-a911-46f0-9c15-4983e911093e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:47.787170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01184dd-a911-46f0-9c15-4983e911093e None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:47.787462 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01184dd-a911-46f0-9c15-4983e911093e None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:47.787745 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01184dd-a911-46f0-9c15-4983e911093e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:47.790097 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01184dd-a911-46f0-9c15-4983e911093e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:47.792013 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01184dd-a911-46f0-9c15-4983e911093e None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:47.794526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 24/45] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:51:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:47.799480 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d60cea-4248-4948-b7ca-4ba6cdecef39 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:47.866235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:47.911947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d60cea-4248-4948-b7ca-4ba6cdecef39 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:47.947234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbafcd-b78d-4c7c-943a-fe10ce9c5dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:47.948464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbafcd-b78d-4c7c-943a-fe10ce9c5dd3 None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:47.948890 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbafcd-b78d-4c7c-943a-fe10ce9c5dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:47.949185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbafcd-b78d-4c7c-943a-fe10ce9c5dd3 None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:47.949925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbafcd-b78d-4c7c-943a-fe10ce9c5dd3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:47.951422 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbafcd-b78d-4c7c-943a-fe10ce9c5dd3 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:47.983328 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 22/46] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Jul 12 21:51:47 2022] POST /identity/v3/projects => generated 312 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:51:49.443495 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0813e9-e38d-40f8-80c6-1498eb2b5b5f None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:49.444014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0813e9-e38d-40f8-80c6-1498eb2b5b5f None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:49.444095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0813e9-e38d-40f8-80c6-1498eb2b5b5f None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:49.445530 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 25/47] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:51:49.451055 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac45b5b-a73e-4b3d-9e82-412ce7231b5f None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:49.451458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac45b5b-a73e-4b3d-9e82-412ce7231b5f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:49.451626 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac45b5b-a73e-4b3d-9e82-412ce7231b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:49.469246 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-4ac45b5b-a73e-4b3d-9e82-412ce7231b5f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:49.470777 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4ac45b5b-a73e-4b3d-9e82-412ce7231b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:49.537416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 23/48] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:51:49.560565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0813e9-e38d-40f8-80c6-1498eb2b5b5f None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:49.675368 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0813e9-e38d-40f8-80c6-1498eb2b5b5f None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:49.688564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:49.710758 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9704642-1e91-43f5-92d2-fa557cf4d95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:49.711486 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9704642-1e91-43f5-92d2-fa557cf4d95b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:49.711777 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9704642-1e91-43f5-92d2-fa557cf4d95b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:49.712059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9704642-1e91-43f5-92d2-fa557cf4d95b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:49.723149 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9704642-1e91-43f5-92d2-fa557cf4d95b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:51:49.731042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-c9704642-1e91-43f5-92d2-fa557cf4d95b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:51:49.731042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:51:49.731042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:51:49.731042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:51:49.731042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:51:49.731042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:51:49.731042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:51:49.731042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:51:49.731042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:51:49.731042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:51:49.731042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:51:49.731042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:51:49.731042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:51:49.731042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:51:49.731042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:51:49.731042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:51:49.731894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:51:49.731894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:51:49.731894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:51:49.731894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:51:49.731894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:51:49.731894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:51:49.731894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:51:49.731894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:51:49.731894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:51:49.731894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:51:49.731894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:51:49.731894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:51:49.731894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:51:49.731894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:51:49.731894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:51:49.731894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:51:49.732634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:51:49.732634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:51:49.732634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:51:49.732634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:51:49.732634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:51:49.732634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:51:49.732634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:51:49.732634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:51:49.732634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:51:49.732634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:51:49.732634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:51:49.732634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:51:49.732634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 26/49] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:51:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:51:49.739339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac45b5b-a73e-4b3d-9e82-412ce7231b5f None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:49.781844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:49.842741 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac45b5b-a73e-4b3d-9e82-412ce7231b5f None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:49.877946 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603bd91-bc6d-43c7-98e9-1ec99b007387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:49.878625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603bd91-bc6d-43c7-98e9-1ec99b007387 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:49.878916 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603bd91-bc6d-43c7-98e9-1ec99b007387 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:49.879198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603bd91-bc6d-43c7-98e9-1ec99b007387 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:49.879967 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603bd91-bc6d-43c7-98e9-1ec99b007387 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:51:49.880284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603bd91-bc6d-43c7-98e9-1ec99b007387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:49.881733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603bd91-bc6d-43c7-98e9-1ec99b007387 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:49.891545 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 24/50] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:51:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:49.896952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9704642-1e91-43f5-92d2-fa557cf4d95b None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:49.998303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9704642-1e91-43f5-92d2-fa557cf4d95b None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:50.032445 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0afd7-a4cf-4edf-b2ab-0e1954d381a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:50.033384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0afd7-a4cf-4edf-b2ab-0e1954d381a7 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:50.033673 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0afd7-a4cf-4edf-b2ab-0e1954d381a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:50.033955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0afd7-a4cf-4edf-b2ab-0e1954d381a7 None admin] PATH_INFO: `/v3/users/a08b779b7e9045028b1b3a86939d306f` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:50.035245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0afd7-a4cf-4edf-b2ab-0e1954d381a7 None admin] RBAC: Authorizing `identity:get_user(user_id=a08b779b7e9045028b1b3a86939d306f)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:50.036782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0afd7-a4cf-4edf-b2ab-0e1954d381a7 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:50.038517 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 27/51] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:51:49 2022] GET /identity/v3/users/a08b779b7e9045028b1b3a86939d306f => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:50.044311 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603bd91-bc6d-43c7-98e9-1ec99b007387 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:50.084806 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47209689-1b40-4f37-a6cc-a5483895e29e None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:50.085100 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47209689-1b40-4f37-a6cc-a5483895e29e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:50.085383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47209689-1b40-4f37-a6cc-a5483895e29e None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:50.086750 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 28/52] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:51:50.092331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e598927-aebd-47b2-8d31-286271f456fd None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:50.092626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e598927-aebd-47b2-8d31-286271f456fd None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:50.092946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e598927-aebd-47b2-8d31-286271f456fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:50.109588 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-8e598927-aebd-47b2-8d31-286271f456fd None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:50.110812 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8e598927-aebd-47b2-8d31-286271f456fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:50.146274 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:50.146675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603bd91-bc6d-43c7-98e9-1ec99b007387 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:50.148258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:50.179853 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 29/53] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:51:50.180842 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f8c33-d41e-40fb-b448-376060aa8748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:50.182270 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f8c33-d41e-40fb-b448-376060aa8748 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:50.182562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f8c33-d41e-40fb-b448-376060aa8748 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:50.182840 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f8c33-d41e-40fb-b448-376060aa8748 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:50.184059 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f8c33-d41e-40fb-b448-376060aa8748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:50.185579 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f8c33-d41e-40fb-b448-376060aa8748 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:50.187209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 25/54] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:51:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:50.193357 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e598927-aebd-47b2-8d31-286271f456fd None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:50.204204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f8c33-d41e-40fb-b448-376060aa8748 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:50.296014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e598927-aebd-47b2-8d31-286271f456fd None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:50.308423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f8c33-d41e-40fb-b448-376060aa8748 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:50.331609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c8035-9a2b-4b0c-b13a-431c1f9beaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:50.332276 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c8035-9a2b-4b0c-b13a-431c1f9beaf9 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:50.332568 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c8035-9a2b-4b0c-b13a-431c1f9beaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:50.332880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c8035-9a2b-4b0c-b13a-431c1f9beaf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:50.333807 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c8035-9a2b-4b0c-b13a-431c1f9beaf9 None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.domain.id=default, user.id=a08b779b7e9045028b1b3a86939d306f) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:51:50.334131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c8035-9a2b-4b0c-b13a-431c1f9beaf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:50.335608 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c8035-9a2b-4b0c-b13a-431c1f9beaf9 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:50.345367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 30/55] 10.222.0.44 () {56 vars in 1322 bytes} [Tue Jul 12 21:51:50 2022] GET /identity/v3/role_assignments?user.id=a08b779b7e9045028b1b3a86939d306f&scope.domain.id=default&role.id=053d1730ed764973abb1490894e07681 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:50.347574 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce6bd4-e689-45a9-b47e-ce16120fffd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:50.348382 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce6bd4-e689-45a9-b47e-ce16120fffd6 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:50.348843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce6bd4-e689-45a9-b47e-ce16120fffd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:50.349004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce6bd4-e689-45a9-b47e-ce16120fffd6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:50.350237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce6bd4-e689-45a9-b47e-ce16120fffd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:50.351726 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce6bd4-e689-45a9-b47e-ce16120fffd6 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:50.358543 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 26/56] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:51:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:51:50.359801 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c8035-9a2b-4b0c-b13a-431c1f9beaf9 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:50.397518 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:50.577142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c8035-9a2b-4b0c-b13a-431c1f9beaf9 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:50.610512 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b570b-9980-4211-b00f-8d58be48fd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:50.611182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b570b-9980-4211-b00f-8d58be48fd16 None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:50.611477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b570b-9980-4211-b00f-8d58be48fd16 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:50.611759 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b570b-9980-4211-b00f-8d58be48fd16 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:50.612469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b570b-9980-4211-b00f-8d58be48fd16 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:50.613978 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b570b-9980-4211-b00f-8d58be48fd16 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:50.639235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 31/57] 10.222.0.44 () {60 vars in 1140 bytes} [Tue Jul 12 21:51:50 2022] POST /identity/v3/projects => generated 298 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:51:52.663981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f673c-43be-475f-88f0-d4ea592414ec None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:52.664499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f673c-43be-475f-88f0-d4ea592414ec None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:52.664582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f673c-43be-475f-88f0-d4ea592414ec None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:52.666002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 27/58] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:51:52.670988 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347f420-4b2b-447b-b7c6-3ed156d804a3 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:52.671387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347f420-4b2b-447b-b7c6-3ed156d804a3 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:52.671554 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347f420-4b2b-447b-b7c6-3ed156d804a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:52.687959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-e347f420-4b2b-447b-b7c6-3ed156d804a3 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:52.689197 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e347f420-4b2b-447b-b7c6-3ed156d804a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:52.755845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 32/59] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:51:52.780046 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f673c-43be-475f-88f0-d4ea592414ec None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:52.822348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:52.881206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f673c-43be-475f-88f0-d4ea592414ec None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:52.915232 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2583b1-2670-4203-9bf9-dc7e1f2521fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:52.916616 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2583b1-2670-4203-9bf9-dc7e1f2521fd None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:52.917145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2583b1-2670-4203-9bf9-dc7e1f2521fd None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:52.917595 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2583b1-2670-4203-9bf9-dc7e1f2521fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:52.918917 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2583b1-2670-4203-9bf9-dc7e1f2521fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:52.920381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2583b1-2670-4203-9bf9-dc7e1f2521fd None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:52.922051 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 28/60] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:51:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:52.927581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347f420-4b2b-447b-b7c6-3ed156d804a3 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:52.945316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:53.026544 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347f420-4b2b-447b-b7c6-3ed156d804a3 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:53.059498 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192c865-3146-4908-b22b-aec4e9626b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:53.060162 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192c865-3146-4908-b22b-aec4e9626b40 None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:53.060450 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192c865-3146-4908-b22b-aec4e9626b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:53.060747 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192c865-3146-4908-b22b-aec4e9626b40 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:53.061454 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192c865-3146-4908-b22b-aec4e9626b40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:53.062906 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192c865-3146-4908-b22b-aec4e9626b40 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:53.087499 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 33/61] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:51:52 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:51:56.262445 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371462c-e449-421a-8adc-c734b240c50e None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:56.263346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371462c-e449-421a-8adc-c734b240c50e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:56.263783 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371462c-e449-421a-8adc-c734b240c50e None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:56.265376 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 29/62] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:51:56.270577 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfb735-8430-4e4c-8f57-aec582a77144 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:56.271335 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfb735-8430-4e4c-8f57-aec582a77144 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:56.271780 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfb735-8430-4e4c-8f57-aec582a77144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:56.297923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-c4dfb735-8430-4e4c-8f57-aec582a77144 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:56.299373 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c4dfb735-8430-4e4c-8f57-aec582a77144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:56.379852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546faad-b366-4ad0-af0e-2a79e92254d2 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:56.380379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546faad-b366-4ad0-af0e-2a79e92254d2 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:56.380917 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546faad-b366-4ad0-af0e-2a79e92254d2 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:56.382469 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 30/63] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:51:56.391488 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 34/64] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:51:56.395820 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f51104-3b8b-45e8-b354-5a797f092dc6 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:56.396119 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f51104-3b8b-45e8-b354-5a797f092dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:56.396403 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f51104-3b8b-45e8-b354-5a797f092dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:56.413737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-a7f51104-3b8b-45e8-b354-5a797f092dc6 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:56.414955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a7f51104-3b8b-45e8-b354-5a797f092dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:56.419482 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe81782-f387-46cf-8669-efbd4319e580 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:56.419770 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe81782-f387-46cf-8669-efbd4319e580 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:56.420053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe81782-f387-46cf-8669-efbd4319e580 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:56.421465 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 35/65] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:51:56.426484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f50b3-5bbc-4fb2-99fd-1e3e623aa06d None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:56.426782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f50b3-5bbc-4fb2-99fd-1e3e623aa06d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:56.427072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f50b3-5bbc-4fb2-99fd-1e3e623aa06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:56.433616 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:56.449212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-a77f50b3-5bbc-4fb2-99fd-1e3e623aa06d None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:56.451627 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a77f50b3-5bbc-4fb2-99fd-1e3e623aa06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:56.498006 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 31/66] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:51:56.505767 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f51104-3b8b-45e8-b354-5a797f092dc6 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:56.525182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 36/67] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:51:56.548512 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f50b3-5bbc-4fb2-99fd-1e3e623aa06d None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:56.561351 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:56.631333 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f51104-3b8b-45e8-b354-5a797f092dc6 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:56.656299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f50b3-5bbc-4fb2-99fd-1e3e623aa06d None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:56.667236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109e0dc-16e4-44e0-a097-dca558b74b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:56.668374 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109e0dc-16e4-44e0-a097-dca558b74b85 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:56.668881 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109e0dc-16e4-44e0-a097-dca558b74b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:56.669332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109e0dc-16e4-44e0-a097-dca558b74b85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:56.680617 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109e0dc-16e4-44e0-a097-dca558b74b85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:51:56.689354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-a109e0dc-16e4-44e0-a097-dca558b74b85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:51:56.689354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:51:56.689354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:51:56.689354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:51:56.689354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:51:56.689354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:51:56.689354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:51:56.689354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:51:56.689354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:51:56.689354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:51:56.689354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:51:56.689354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:51:56.689354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:51:56.689354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:51:56.689354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:51:56.689354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:51:56.690221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:51:56.690221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:51:56.690221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:51:56.690221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:51:56.690221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:51:56.690221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:51:56.690221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:51:56.690221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:51:56.690221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:51:56.690221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:51:56.690221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:51:56.690221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:51:56.690221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:51:56.690221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:51:56.690221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:51:56.690221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:51:56.691234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:51:56.691234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:51:56.691234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:51:56.691234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:51:56.691234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:51:56.691234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:51:56.691234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:51:56.691234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:51:56.691234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:51:56.691234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:51:56.691234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:51:56.691234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:51:56.693503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 32/68] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:51:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:51:56.694246 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:56.694667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109e0dc-16e4-44e0-a097-dca558b74b85 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:56.700753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3bcd7-4916-430a-84f2-0598bff781c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:56.701672 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3bcd7-4916-430a-84f2-0598bff781c7 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:56.702073 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3bcd7-4916-430a-84f2-0598bff781c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:56.702447 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3bcd7-4916-430a-84f2-0598bff781c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:56.714386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3bcd7-4916-430a-84f2-0598bff781c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:51:56.722748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-c6b3bcd7-4916-430a-84f2-0598bff781c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:51:56.722748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:51:56.722748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:51:56.722748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:51:56.722748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:51:56.722748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:51:56.722748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:51:56.722748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:51:56.722748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:51:56.722748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:51:56.722748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:51:56.722748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:51:56.722748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:51:56.722748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:51:56.722748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:51:56.722748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:51:56.723613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:51:56.723613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:51:56.723613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:51:56.723613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:51:56.723613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:51:56.723613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:51:56.723613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:51:56.723613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:51:56.723613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:51:56.723613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:51:56.723613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:51:56.723613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:51:56.723613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:51:56.723613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:51:56.723613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:51:56.723613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:51:56.724358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:51:56.724358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:51:56.724358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:51:56.724358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:51:56.724358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:51:56.724358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:51:56.724358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:51:56.724358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:51:56.724358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:51:56.724358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:51:56.724358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:51:56.724358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:51:56.726798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 37/69] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:51:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:51:56.727792 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3bcd7-4916-430a-84f2-0598bff781c7 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:56.796600 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:56.805902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109e0dc-16e4-44e0-a097-dca558b74b85 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:56.847833 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554e021-c54d-415b-bb35-b83214a41259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:56.848659 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554e021-c54d-415b-bb35-b83214a41259 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:56.848829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3bcd7-4916-430a-84f2-0598bff781c7 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:56.848976 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554e021-c54d-415b-bb35-b83214a41259 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:56.849254 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554e021-c54d-415b-bb35-b83214a41259 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:56.860345 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554e021-c54d-415b-bb35-b83214a41259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:51:56.868508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-0554e021-c54d-415b-bb35-b83214a41259 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:51:56.868508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:51:56.868508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:51:56.868508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:51:56.868508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:51:56.868508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:51:56.868508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:51:56.868508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:51:56.868508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:51:56.868508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:51:56.868508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:51:56.868508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:51:56.868508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:51:56.868508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:51:56.868508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:51:56.868508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:51:56.875614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:51:56.875614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:51:56.875614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:51:56.875614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:51:56.875614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:51:56.875614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:51:56.875614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:51:56.875614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:51:56.875614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:51:56.875614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:51:56.875614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:51:56.875614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:51:56.875614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:51:56.875614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:51:56.875614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:51:56.875614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:51:56.876408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:51:56.876408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:51:56.876408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:51:56.876408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:51:56.876408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:51:56.876408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:51:56.876408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:51:56.876408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:51:56.876408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:51:56.876408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:51:56.876408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:51:56.876408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:51:56.876408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 33/70] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:51:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:51:56.876408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554e021-c54d-415b-bb35-b83214a41259 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:56.887675 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22651b2-fcac-44e0-aa3d-1f011783f8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:56.888800 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22651b2-fcac-44e0-aa3d-1f011783f8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:56.889302 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22651b2-fcac-44e0-aa3d-1f011783f8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:56.890269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22651b2-fcac-44e0-aa3d-1f011783f8c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:56.891296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22651b2-fcac-44e0-aa3d-1f011783f8c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:51:56.892186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22651b2-fcac-44e0-aa3d-1f011783f8c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:56.894421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22651b2-fcac-44e0-aa3d-1f011783f8c7 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:56.905676 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 38/71] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:51:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:56.907340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517090e3-9941-4fd9-8ba5-071027ef09a0 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:56.907635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517090e3-9941-4fd9-8ba5-071027ef09a0 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:56.907916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517090e3-9941-4fd9-8ba5-071027ef09a0 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:56.909496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 39/72] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:51:56.909951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517090e3-9941-4fd9-8ba5-071027ef09a0 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:56.975089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:56.993737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554e021-c54d-415b-bb35-b83214a41259 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:57.021479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517090e3-9941-4fd9-8ba5-071027ef09a0 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:57.028279 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe869f14-7f6f-4713-8560-62abeb047d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:57.029050 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869f14-7f6f-4713-8560-62abeb047d1b None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:57.029338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869f14-7f6f-4713-8560-62abeb047d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:57.029648 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869f14-7f6f-4713-8560-62abeb047d1b None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:57.030395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe869f14-7f6f-4713-8560-62abeb047d1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:51:57.030703 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe869f14-7f6f-4713-8560-62abeb047d1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:57.032104 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe869f14-7f6f-4713-8560-62abeb047d1b None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:57.040252 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 34/73] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:51:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:57.041611 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe869f14-7f6f-4713-8560-62abeb047d1b None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:57.056114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33feea-929e-45eb-959f-461631d638f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:57.056802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33feea-929e-45eb-959f-461631d638f4 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:57.057092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33feea-929e-45eb-959f-461631d638f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:57.057459 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33feea-929e-45eb-959f-461631d638f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:57.058217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33feea-929e-45eb-959f-461631d638f4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:51:57.058532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33feea-929e-45eb-959f-461631d638f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:57.059918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33feea-929e-45eb-959f-461631d638f4 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:57.061570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:57.068640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 40/74] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:51:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:57.069931 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0998d8-0675-4c0e-b2ba-884105ae6139 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:57.070228 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0998d8-0675-4c0e-b2ba-884105ae6139 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:57.070515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0998d8-0675-4c0e-b2ba-884105ae6139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:57.087353 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-2d0998d8-0675-4c0e-b2ba-884105ae6139 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:57.088823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2d0998d8-0675-4c0e-b2ba-884105ae6139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:57.144734 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe869f14-7f6f-4713-8560-62abeb047d1b None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:57.155135 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 41/75] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:51:57.156301 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7ce82-a942-4cd0-bd97-0a44096ab183 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:57.156596 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7ce82-a942-4cd0-bd97-0a44096ab183 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:57.156915 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7ce82-a942-4cd0-bd97-0a44096ab183 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:57.158349 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 42/76] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:51:57.162839 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7ce82-a942-4cd0-bd97-0a44096ab183 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:57.184240 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69f642-d67f-44c0-ad59-1a51caa64a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:57.185173 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69f642-d67f-44c0-ad59-1a51caa64a91 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:57.185462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69f642-d67f-44c0-ad59-1a51caa64a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:57.185745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69f642-d67f-44c0-ad59-1a51caa64a91 None admin] PATH_INFO: `/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:57.204145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c69f642-d67f-44c0-ad59-1a51caa64a91 None admin] RBAC: Authorizing `identity:get_user(user_id=f0a92aa0e8164ee0890c4bb9a039faf7)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:57.205667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c69f642-d67f-44c0-ad59-1a51caa64a91 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:57.207427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 35/77] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:51:57 2022] GET /identity/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:57.208257 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69f642-d67f-44c0-ad59-1a51caa64a91 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:57.222694 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:57.279864 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7ce82-a942-4cd0-bd97-0a44096ab183 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:57.286959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:57.313079 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69f642-d67f-44c0-ad59-1a51caa64a91 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:57.318843 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f680f67-aa7c-4a0c-bd89-34c2188350f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:57.319711 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f680f67-aa7c-4a0c-bd89-34c2188350f2 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:57.319989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f680f67-aa7c-4a0c-bd89-34c2188350f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:57.320289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f680f67-aa7c-4a0c-bd89-34c2188350f2 None admin] PATH_INFO: `/v3/users/a08b779b7e9045028b1b3a86939d306f` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:57.321472 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f680f67-aa7c-4a0c-bd89-34c2188350f2 None admin] RBAC: Authorizing `identity:get_user(user_id=a08b779b7e9045028b1b3a86939d306f)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:57.323476 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f680f67-aa7c-4a0c-bd89-34c2188350f2 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:57.325351 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 43/78] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:51:57 2022] GET /identity/v3/users/a08b779b7e9045028b1b3a86939d306f => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:57.326474 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c680c-9326-4e31-97dd-feb402a38631 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:57.326761 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c680c-9326-4e31-97dd-feb402a38631 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:57.327160 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c680c-9326-4e31-97dd-feb402a38631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:57.344920 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-9e8c680c-9326-4e31-97dd-feb402a38631 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:51:57.346108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-9e8c680c-9326-4e31-97dd-feb402a38631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:51:57.349711 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c60e29-1678-4252-8722-c64a557dbd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:57.350604 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c60e29-1678-4252-8722-c64a557dbd58 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:57.350891 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c60e29-1678-4252-8722-c64a557dbd58 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:57.351171 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c60e29-1678-4252-8722-c64a557dbd58 None admin] PATH_INFO: `/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:57.352380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c60e29-1678-4252-8722-c64a557dbd58 None admin] RBAC: Authorizing `identity:get_user(user_id=f0a92aa0e8164ee0890c4bb9a039faf7)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:57.354289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c60e29-1678-4252-8722-c64a557dbd58 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:57.355960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 36/79] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:51:57 2022] GET /identity/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:57.356744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c60e29-1678-4252-8722-c64a557dbd58 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:57.413741 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 44/80] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:51:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:51:57.414657 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c680c-9326-4e31-97dd-feb402a38631 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:57.457108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c60e29-1678-4252-8722-c64a557dbd58 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:57.466960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:57.493992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80d7c9-5ea1-4c12-847a-273dc8e40269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:57.495439 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80d7c9-5ea1-4c12-847a-273dc8e40269 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:57.495656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80d7c9-5ea1-4c12-847a-273dc8e40269 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:57.495935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80d7c9-5ea1-4c12-847a-273dc8e40269 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:57.509682 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80d7c9-5ea1-4c12-847a-273dc8e40269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:51:57.517935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-8e80d7c9-5ea1-4c12-847a-273dc8e40269 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:51:57.517935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:51:57.517935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:51:57.517935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:51:57.517935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:51:57.517935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:51:57.517935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:51:57.517935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:51:57.517935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:51:57.517935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:51:57.517935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:51:57.517935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:51:57.517935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:51:57.517935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:51:57.517935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:51:57.517935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:51:57.518796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:51:57.518796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:51:57.518796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:51:57.518796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:51:57.518796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:51:57.518796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:51:57.518796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:51:57.518796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:51:57.518796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:51:57.518796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:51:57.518796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:51:57.518796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:51:57.518796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:51:57.518796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:51:57.518796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:51:57.518796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:51:57.519541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:51:57.519541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:51:57.519541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:51:57.519541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:51:57.519541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:51:57.519541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:51:57.519541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:51:57.519541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:51:57.519541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:51:57.519541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:51:57.519541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:51:57.519541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:51:57.519541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 37/81] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:51:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:51:57.520162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80d7c9-5ea1-4c12-847a-273dc8e40269 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:57.528447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:57.531465 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c680c-9326-4e31-97dd-feb402a38631 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:57.568356 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8e6f7-7d17-4976-a461-3eaa8bef1ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:57.568948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8e6f7-7d17-4976-a461-3eaa8bef1ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:57.569233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8e6f7-7d17-4976-a461-3eaa8bef1ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:57.569513 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8e6f7-7d17-4976-a461-3eaa8bef1ee3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:57.631230 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80d7c9-5ea1-4c12-847a-273dc8e40269 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:57.640919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8e6f7-7d17-4976-a461-3eaa8bef1ee3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:57.642408 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8e6f7-7d17-4976-a461-3eaa8bef1ee3 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:57.665463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93091915-c0cb-4269-a73f-366dd5548b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:57.666696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93091915-c0cb-4269-a73f-366dd5548b7a None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:57.666980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93091915-c0cb-4269-a73f-366dd5548b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:57.667264 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93091915-c0cb-4269-a73f-366dd5548b7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:57.686983 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:57.736168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93091915-c0cb-4269-a73f-366dd5548b7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:57.737690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93091915-c0cb-4269-a73f-366dd5548b7a None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:57.740316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:57.746618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 45/82] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:51:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:51:57.747745 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8e6f7-7d17-4976-a461-3eaa8bef1ee3 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:57.843058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 38/83] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:51:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:51:57.843848 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93091915-c0cb-4269-a73f-366dd5548b7a None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:57.850523 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8e6f7-7d17-4976-a461-3eaa8bef1ee3 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:57.888170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e85f0-88ce-4630-808f-03dbf77563c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:57.888886 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e85f0-88ce-4630-808f-03dbf77563c1 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:57.889150 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e85f0-88ce-4630-808f-03dbf77563c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:57.889434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e85f0-88ce-4630-808f-03dbf77563c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:57.904836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:57.946197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93091915-c0cb-4269-a73f-366dd5548b7a None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:57.949196 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:57.961458 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e85f0-88ce-4630-808f-03dbf77563c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:57.963951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e85f0-88ce-4630-808f-03dbf77563c1 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:57.987822 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952ace2-cf24-45aa-a839-201297bbc448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:57.988624 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952ace2-cf24-45aa-a839-201297bbc448 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:57.988950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952ace2-cf24-45aa-a839-201297bbc448 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:57.989239 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952ace2-cf24-45aa-a839-201297bbc448 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:57.990417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952ace2-cf24-45aa-a839-201297bbc448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:57.991893 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952ace2-cf24-45aa-a839-201297bbc448 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:57.993569 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 39/84] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:51:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:57.994599 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952ace2-cf24-45aa-a839-201297bbc448 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:58.063452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 46/85] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:51:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:51:58.064307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e85f0-88ce-4630-808f-03dbf77563c1 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:58.096297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952ace2-cf24-45aa-a839-201297bbc448 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:58.108861 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:58.130777 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d8c9e-ad22-41b0-8c26-9afbc863cffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:58.131447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d8c9e-ad22-41b0-8c26-9afbc863cffe None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:58.131732 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d8c9e-ad22-41b0-8c26-9afbc863cffe None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:58.132012 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d8c9e-ad22-41b0-8c26-9afbc863cffe None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:58.132789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d8c9e-ad22-41b0-8c26-9afbc863cffe None admin] RBAC: Adding query filter params (name=member) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:51:58.133103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d8c9e-ad22-41b0-8c26-9afbc863cffe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:58.134559 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d8c9e-ad22-41b0-8c26-9afbc863cffe None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:58.143030 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 40/86] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:51:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:58.143842 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d8c9e-ad22-41b0-8c26-9afbc863cffe None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:58.164680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e85f0-88ce-4630-808f-03dbf77563c1 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:58.178012 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:58.200833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af929e09-3fad-4073-8da8-1777103c8027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:58.201693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af929e09-3fad-4073-8da8-1777103c8027 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:58.201975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af929e09-3fad-4073-8da8-1777103c8027 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:58.202255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af929e09-3fad-4073-8da8-1777103c8027 None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:58.203423 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af929e09-3fad-4073-8da8-1777103c8027 None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:58.204927 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af929e09-3fad-4073-8da8-1777103c8027 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:58.206629 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 47/87] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:51:58 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:58.207372 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af929e09-3fad-4073-8da8-1777103c8027 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:58.247369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d8c9e-ad22-41b0-8c26-9afbc863cffe None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:58.282348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab631f8-17f5-4d25-88ec-946442c71c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:58.283215 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab631f8-17f5-4d25-88ec-946442c71c04 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:58.283510 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab631f8-17f5-4d25-88ec-946442c71c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:58.283789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab631f8-17f5-4d25-88ec-946442c71c04 None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:58.285023 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab631f8-17f5-4d25-88ec-946442c71c04 None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:58.286497 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab631f8-17f5-4d25-88ec-946442c71c04 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:58.288371 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 41/88] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:51:58 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:58.289096 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab631f8-17f5-4d25-88ec-946442c71c04 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:58.311940 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af929e09-3fad-4073-8da8-1777103c8027 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:58.325997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:58.347386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e57b9-331a-4b72-b45e-4a282065a818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:58.347986 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e57b9-331a-4b72-b45e-4a282065a818 None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:58.348375 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e57b9-331a-4b72-b45e-4a282065a818 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:58.348559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e57b9-331a-4b72-b45e-4a282065a818 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:58.349317 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e57b9-331a-4b72-b45e-4a282065a818 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:58.350771 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e57b9-331a-4b72-b45e-4a282065a818 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:58.370518 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 48/89] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:51:58 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:51:58.371392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e57b9-331a-4b72-b45e-4a282065a818 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:58.393041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab631f8-17f5-4d25-88ec-946442c71c04 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:58.394808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:58.433965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02826bc6-5690-4ee4-a767-beaaa4f0c038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:58.435380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02826bc6-5690-4ee4-a767-beaaa4f0c038 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:58.435685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02826bc6-5690-4ee4-a767-beaaa4f0c038 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:58.435970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02826bc6-5690-4ee4-a767-beaaa4f0c038 None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:58.437628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02826bc6-5690-4ee4-a767-beaaa4f0c038 None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:58.439168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02826bc6-5690-4ee4-a767-beaaa4f0c038 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:58.441281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 42/90] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:51:58 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:58.441281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02826bc6-5690-4ee4-a767-beaaa4f0c038 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:58.508454 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e57b9-331a-4b72-b45e-4a282065a818 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:58.551884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f22926-f77f-4170-a55d-ea315888602f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:58.553071 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f22926-f77f-4170-a55d-ea315888602f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:58.553363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f22926-f77f-4170-a55d-ea315888602f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:58.553646 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f22926-f77f-4170-a55d-ea315888602f None admin] PATH_INFO: `/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:58.555055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f22926-f77f-4170-a55d-ea315888602f None admin] RBAC: Authorizing `identity:get_user(user_id=f0a92aa0e8164ee0890c4bb9a039faf7)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:58.557135 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f22926-f77f-4170-a55d-ea315888602f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:58.558830 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 49/91] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:51:58 2022] GET /identity/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:58.559595 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f22926-f77f-4170-a55d-ea315888602f None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:58.561714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02826bc6-5690-4ee4-a767-beaaa4f0c038 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:58.601313 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9045bf-721e-46f8-a6b7-afd9860129c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:58.601886 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9045bf-721e-46f8-a6b7-afd9860129c0 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:58.603142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9045bf-721e-46f8-a6b7-afd9860129c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:58.603947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9045bf-721e-46f8-a6b7-afd9860129c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:58.605776 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9045bf-721e-46f8-a6b7-afd9860129c0 None admin] RBAC: Adding query filter params (role.id=b5ee21738ecb4079977d078d50ddbcf9, scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5, user.id=f0a92aa0e8164ee0890c4bb9a039faf7) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:51:58.605776 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9045bf-721e-46f8-a6b7-afd9860129c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:58.607039 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9045bf-721e-46f8-a6b7-afd9860129c0 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:58.612985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:58.613975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 43/92] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:51:58 2022] GET /identity/v3/role_assignments?user.id=f0a92aa0e8164ee0890c4bb9a039faf7&scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5&role.id=b5ee21738ecb4079977d078d50ddbcf9 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:58.614812 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9045bf-721e-46f8-a6b7-afd9860129c0 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:58.668521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:58.708596 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f22926-f77f-4170-a55d-ea315888602f None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:58.761503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c2534-98a6-4617-b39b-f2ca4997ea0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:58.762236 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c2534-98a6-4617-b39b-f2ca4997ea0b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:58.762531 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c2534-98a6-4617-b39b-f2ca4997ea0b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:58.762813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c2534-98a6-4617-b39b-f2ca4997ea0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:58.763678 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c2534-98a6-4617-b39b-f2ca4997ea0b None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5, user.id=a08b779b7e9045028b1b3a86939d306f) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:51:58.763995 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c2534-98a6-4617-b39b-f2ca4997ea0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:58.765515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c2534-98a6-4617-b39b-f2ca4997ea0b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:58.771923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 50/93] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:51:58 2022] GET /identity/v3/role_assignments?user.id=a08b779b7e9045028b1b3a86939d306f&scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5&role.id=053d1730ed764973abb1490894e07681 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:58.772811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c2534-98a6-4617-b39b-f2ca4997ea0b None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:58.773670 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9045bf-721e-46f8-a6b7-afd9860129c0 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:58.823391 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efe8a9-7fa5-416b-b72f-fbdcec4444e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:58.824084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efe8a9-7fa5-416b-b72f-fbdcec4444e5 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:58.824374 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efe8a9-7fa5-416b-b72f-fbdcec4444e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:58.824657 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efe8a9-7fa5-416b-b72f-fbdcec4444e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:58.825553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efe8a9-7fa5-416b-b72f-fbdcec4444e5 None admin] RBAC: Adding query filter params (role.id=160d4162340e472b9712a7f15cb2ecca, scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5, user.id=f0a92aa0e8164ee0890c4bb9a039faf7) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:51:58.825870 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efe8a9-7fa5-416b-b72f-fbdcec4444e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:58.827814 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efe8a9-7fa5-416b-b72f-fbdcec4444e5 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:58.836138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 44/94] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:51:58 2022] GET /identity/v3/role_assignments?user.id=f0a92aa0e8164ee0890c4bb9a039faf7&scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5&role.id=160d4162340e472b9712a7f15cb2ecca => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:58.921918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:58.926027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c2534-98a6-4617-b39b-f2ca4997ea0b None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:58.981755 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9ccb2-0ecb-4b5c-975a-c106c39a990d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:58.982439 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9ccb2-0ecb-4b5c-975a-c106c39a990d None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:58.982723 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9ccb2-0ecb-4b5c-975a-c106c39a990d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:58.983001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9ccb2-0ecb-4b5c-975a-c106c39a990d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:59.076754 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9ccb2-0ecb-4b5c-975a-c106c39a990d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:59.078256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9ccb2-0ecb-4b5c-975a-c106c39a990d None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:59.191743 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:59.196892 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 51/95] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:51:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:51:59.203751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efe8a9-7fa5-416b-b72f-fbdcec4444e5 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:59.346787 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:51:59.350855 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efe8a9-7fa5-416b-b72f-fbdcec4444e5 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:59.396161 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5284fb2-4fba-4275-bc16-5cdf7faf566b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:59.397067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5284fb2-4fba-4275-bc16-5cdf7faf566b None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:59.397357 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5284fb2-4fba-4275-bc16-5cdf7faf566b None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:59.397639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5284fb2-4fba-4275-bc16-5cdf7faf566b None admin] PATH_INFO: `/v3/projects/1693b44e5fea4067af432358e61ee9f5` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:59.401449 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5284fb2-4fba-4275-bc16-5cdf7faf566b None admin] RBAC: Authorizing `identity:get_project(project_id=1693b44e5fea4067af432358e61ee9f5)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:59.402945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5284fb2-4fba-4275-bc16-5cdf7faf566b None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:59.404737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 45/96] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:51:59 2022] GET /identity/v3/projects/1693b44e5fea4067af432358e61ee9f5 => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:51:59.410603 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9ccb2-0ecb-4b5c-975a-c106c39a990d None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:51:59.560988 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9ccb2-0ecb-4b5c-975a-c106c39a990d None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:51:59.609604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17052f57-c539-45db-8060-820111b34115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:51:59.610273 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17052f57-c539-45db-8060-820111b34115 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:51:59.610561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17052f57-c539-45db-8060-820111b34115 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:51:59.610844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17052f57-c539-45db-8060-820111b34115 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:51:59.611698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17052f57-c539-45db-8060-820111b34115 None admin] RBAC: Adding query filter params (role.id=b5ee21738ecb4079977d078d50ddbcf9, scope.project.id=1693b44e5fea4067af432358e61ee9f5, user.id=f0a92aa0e8164ee0890c4bb9a039faf7) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:51:59.612016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17052f57-c539-45db-8060-820111b34115 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:51:59.613506 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17052f57-c539-45db-8060-820111b34115 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:51:59.625299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 52/97] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:51:59 2022] GET /identity/v3/role_assignments?user.id=f0a92aa0e8164ee0890c4bb9a039faf7&scope.project.id=1693b44e5fea4067af432358e61ee9f5&role.id=b5ee21738ecb4079977d078d50ddbcf9 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:01.505294 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625179b-2fdd-41af-86af-91dab2c679b7 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:01.506811 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625179b-2fdd-41af-86af-91dab2c679b7 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:01.507329 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625179b-2fdd-41af-86af-91dab2c679b7 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:01.509368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 46/98] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:52:01.516421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239fdf9-70f6-4608-ab15-bd0cd505d48d None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:01.517381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239fdf9-70f6-4608-ab15-bd0cd505d48d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:01.517861 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239fdf9-70f6-4608-ab15-bd0cd505d48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:01.540219 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-8239fdf9-70f6-4608-ab15-bd0cd505d48d None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:01.542572 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8239fdf9-70f6-4608-ab15-bd0cd505d48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:01.629468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:01.639342 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 53/99] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:52:01.657059 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076d29c-ac24-4424-912b-37d1ee19fdfb None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:01.657353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076d29c-ac24-4424-912b-37d1ee19fdfb None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:01.657638 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076d29c-ac24-4424-912b-37d1ee19fdfb None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:01.659029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 47/100] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:01.664437 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0975789-325c-478f-935f-8ab17f9f959d None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:01.664773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0975789-325c-478f-935f-8ab17f9f959d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:01.665066 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0975789-325c-478f-935f-8ab17f9f959d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:01.685250 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-a0975789-325c-478f-935f-8ab17f9f959d None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:01.686460 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a0975789-325c-478f-935f-8ab17f9f959d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:01.695224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076d29c-ac24-4424-912b-37d1ee19fdfb None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:01.792458 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 54/101] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:01.794402 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329258de-6abc-4a38-9171-3cfcca3ac18c None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:01.794808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329258de-6abc-4a38-9171-3cfcca3ac18c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:01.795101 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329258de-6abc-4a38-9171-3cfcca3ac18c None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:01.796419 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 55/102] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:01.802038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a92c0-21bf-4079-a6c8-7f5055f030e3 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:01.802336 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a92c0-21bf-4079-a6c8-7f5055f030e3 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:01.802620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a92c0-21bf-4079-a6c8-7f5055f030e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:01.828515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-711a92c0-21bf-4079-a6c8-7f5055f030e3 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:01.829887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-711a92c0-21bf-4079-a6c8-7f5055f030e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:01.905798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076d29c-ac24-4424-912b-37d1ee19fdfb None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:01.914286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 56/103] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:01.915412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a92c0-21bf-4079-a6c8-7f5055f030e3 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:01.949561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb0fe5-383d-4f15-991a-8fbf6711afe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:01.950377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb0fe5-383d-4f15-991a-8fbf6711afe4 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:01.950665 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb0fe5-383d-4f15-991a-8fbf6711afe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:01.950946 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb0fe5-383d-4f15-991a-8fbf6711afe4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:01.965728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb0fe5-383d-4f15-991a-8fbf6711afe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:01.975915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-d5bb0fe5-383d-4f15-991a-8fbf6711afe4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:52:01.975915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:01.975915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:01.975915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:01.975915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:01.975915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:01.975915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:01.975915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:01.975915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:01.975915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:01.975915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:01.975915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:01.975915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:01.975915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:01.975915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:01.975915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:01.976817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:01.976817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:01.976817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:01.976817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:01.976817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:01.976817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:01.976817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:01.976817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:01.976817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:01.976817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:01.976817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:01.976817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:01.976817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:01.976817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:01.976817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:01.976817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:01.977601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:01.977601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:01.977601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:01.977601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:01.977601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:01.977601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:01.977601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:01.977601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:01.977601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:01.977601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:01.977601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:52:01.977601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:01.977601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 48/104] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:52:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:52:01.979199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ecfca-9583-4c40-95c8-b414c6a5eb9f None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:01.979947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ecfca-9583-4c40-95c8-b414c6a5eb9f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:01.980378 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ecfca-9583-4c40-95c8-b414c6a5eb9f None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:01.982960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 49/105] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:01.988311 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ecfca-9583-4c40-95c8-b414c6a5eb9f None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:02.005038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:02.030319 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:02.055774 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a92c0-21bf-4079-a6c8-7f5055f030e3 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:02.092955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10094a50-7a8c-493f-80f0-4e64be870d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:02.093777 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10094a50-7a8c-493f-80f0-4e64be870d5f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:02.094068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10094a50-7a8c-493f-80f0-4e64be870d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:02.094350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10094a50-7a8c-493f-80f0-4e64be870d5f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:02.105058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10094a50-7a8c-493f-80f0-4e64be870d5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:02.113239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-10094a50-7a8c-493f-80f0-4e64be870d5f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:02.113239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:02.113239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:02.113239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:02.113239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:02.113239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:02.113239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:02.113239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:02.113239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:02.113239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:02.113239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:02.113239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:02.113239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:02.113239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:02.113239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:02.113239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:02.114137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:02.114137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:02.114137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:02.114137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:02.114137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:02.114137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:02.114137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:02.114137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:02.114137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:02.114137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:02.114137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:02.114137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:02.114137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:02.114137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:02.114137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:02.114137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:02.114912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:02.114912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:02.114912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:02.114912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:02.114912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:02.114912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:02.114912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:02.114912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:02.114912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:02.114912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:02.114912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:02.114912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:02.114912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 57/106] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:52:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:02.115541 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10094a50-7a8c-493f-80f0-4e64be870d5f None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:02.125465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ecfca-9583-4c40-95c8-b414c6a5eb9f None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:02.161131 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521498a0-18e5-4643-b7fd-00cca90f8fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:02.161933 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521498a0-18e5-4643-b7fd-00cca90f8fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:02.162221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521498a0-18e5-4643-b7fd-00cca90f8fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:02.162503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521498a0-18e5-4643-b7fd-00cca90f8fb7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:02.174004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521498a0-18e5-4643-b7fd-00cca90f8fb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:02.182366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-521498a0-18e5-4643-b7fd-00cca90f8fb7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:02.182366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:02.182366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:02.182366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:02.182366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:02.182366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:02.182366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:02.182366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:02.182366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:02.182366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:02.182366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:02.182366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:02.182366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:02.182366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:02.182366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:02.182366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:02.183266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:02.183266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:02.183266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:02.183266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:02.183266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:02.183266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:02.183266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:02.183266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:02.183266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:02.183266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:02.183266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:02.183266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:02.183266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:02.183266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:02.183266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:02.183266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:02.184057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:02.184057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:02.184057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:02.184057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:02.184057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:02.184057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:02.184057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:02.184057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:02.184057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:02.184057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:02.184057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:02.184057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:02.184057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 50/107] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:02.185314 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5f140-f30a-443d-ac3b-26fc90243a90 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:02.185314 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5f140-f30a-443d-ac3b-26fc90243a90 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:02.185464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5f140-f30a-443d-ac3b-26fc90243a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:02.203388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-84a5f140-f30a-443d-ac3b-26fc90243a90 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:02.204606 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-84a5f140-f30a-443d-ac3b-26fc90243a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:02.231937 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10094a50-7a8c-493f-80f0-4e64be870d5f None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:02.247326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:02.271208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c449a5-2b5b-40d8-b267-6fa7ecd3361f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:02.272000 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c449a5-2b5b-40d8-b267-6fa7ecd3361f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:02.272354 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c449a5-2b5b-40d8-b267-6fa7ecd3361f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:02.272810 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c449a5-2b5b-40d8-b267-6fa7ecd3361f None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:02.273468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c449a5-2b5b-40d8-b267-6fa7ecd3361f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:02.273784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c449a5-2b5b-40d8-b267-6fa7ecd3361f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:02.275748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c449a5-2b5b-40d8-b267-6fa7ecd3361f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:02.278159 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 51/108] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:02.279380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bb897-87b9-4d11-9219-3b211ee52b72 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:02.279678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bb897-87b9-4d11-9219-3b211ee52b72 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:02.279965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bb897-87b9-4d11-9219-3b211ee52b72 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:02.281469 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 52/109] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:02.285990 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bb897-87b9-4d11-9219-3b211ee52b72 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:02.305638 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 58/110] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:02.306519 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c449a5-2b5b-40d8-b267-6fa7ecd3361f None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:02.311835 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:02.398668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bb897-87b9-4d11-9219-3b211ee52b72 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:02.428105 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c449a5-2b5b-40d8-b267-6fa7ecd3361f None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:02.433572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b14f0d-2621-4df6-9729-1302e117d9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:02.434255 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b14f0d-2621-4df6-9729-1302e117d9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:02.434538 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b14f0d-2621-4df6-9729-1302e117d9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:02.434820 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b14f0d-2621-4df6-9729-1302e117d9c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:02.435572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b14f0d-2621-4df6-9729-1302e117d9c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:02.435883 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b14f0d-2621-4df6-9729-1302e117d9c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:02.437328 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b14f0d-2621-4df6-9729-1302e117d9c6 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:02.445924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 53/111] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:52:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:02.447640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f311a72-f648-43fa-a064-ca83c5e37803 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:02.447928 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f311a72-f648-43fa-a064-ca83c5e37803 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:02.448207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f311a72-f648-43fa-a064-ca83c5e37803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:02.464356 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-4f311a72-f648-43fa-a064-ca83c5e37803 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:02.465570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4f311a72-f648-43fa-a064-ca83c5e37803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:02.471213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305af08-61d6-4fae-adf0-706706710e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:02.471889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305af08-61d6-4fae-adf0-706706710e66 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:02.472172 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305af08-61d6-4fae-adf0-706706710e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:02.472449 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305af08-61d6-4fae-adf0-706706710e66 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:02.473635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305af08-61d6-4fae-adf0-706706710e66 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:02.473635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305af08-61d6-4fae-adf0-706706710e66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:02.474926 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305af08-61d6-4fae-adf0-706706710e66 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:02.484101 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 59/112] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:02.488532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305af08-61d6-4fae-adf0-706706710e66 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:02.538185 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:02.543980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 54/113] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:02.544881 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f311a72-f648-43fa-a064-ca83c5e37803 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:02.563978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:02.617786 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305af08-61d6-4fae-adf0-706706710e66 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:02.654389 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0b069-49ab-40ad-b56d-be4772208d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:02.655190 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0b069-49ab-40ad-b56d-be4772208d23 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:02.655481 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0b069-49ab-40ad-b56d-be4772208d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:02.655764 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0b069-49ab-40ad-b56d-be4772208d23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:02.663275 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f311a72-f648-43fa-a064-ca83c5e37803 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:02.667304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba0b069-49ab-40ad-b56d-be4772208d23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:02.675117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-1ba0b069-49ab-40ad-b56d-be4772208d23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:02.675117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:02.675117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:02.675117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:02.675117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:02.675117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:02.675117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:02.675117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:02.675117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:02.675117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:02.675117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:02.675117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:02.675117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:02.675117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:02.675117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:02.675117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:02.676063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:02.676063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:02.676063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:02.676063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:02.676063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:02.676063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:02.676063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:02.676063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:02.676063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:02.676063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:02.676063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:02.676063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:02.676063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:02.676063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:02.676063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:02.676063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:02.676857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:02.676857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:02.676857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:02.676857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:02.676857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:02.676857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:02.676857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:02.676857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:02.676857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:02.676857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:02.676857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:02.676857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:02.676857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 60/114] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:02.677490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0b069-49ab-40ad-b56d-be4772208d23 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:02.705990 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7179bc7-baca-41cc-b335-2d268273cd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:02.706884 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7179bc7-baca-41cc-b335-2d268273cd4e None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:02.707171 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7179bc7-baca-41cc-b335-2d268273cd4e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:02.707444 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7179bc7-baca-41cc-b335-2d268273cd4e None admin] PATH_INFO: `/v3/users/a08b779b7e9045028b1b3a86939d306f` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:02.708680 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7179bc7-baca-41cc-b335-2d268273cd4e None admin] RBAC: Authorizing `identity:get_user(user_id=a08b779b7e9045028b1b3a86939d306f)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:02.710176 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7179bc7-baca-41cc-b335-2d268273cd4e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:02.711839 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 55/115] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:02 2022] GET /identity/v3/users/a08b779b7e9045028b1b3a86939d306f => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:02.716903 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7179bc7-baca-41cc-b335-2d268273cd4e None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:02.784142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:02.788492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0b069-49ab-40ad-b56d-be4772208d23 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:02.801081 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:02.826622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7179bc7-baca-41cc-b335-2d268273cd4e None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:02.828846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158fcb2-b3ad-4040-8a8e-401b0ad14805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:02.829875 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158fcb2-b3ad-4040-8a8e-401b0ad14805 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:02.830166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158fcb2-b3ad-4040-8a8e-401b0ad14805 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:02.830449 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158fcb2-b3ad-4040-8a8e-401b0ad14805 None admin] PATH_INFO: `/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:02.831679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158fcb2-b3ad-4040-8a8e-401b0ad14805 None admin] RBAC: Authorizing `identity:get_user(user_id=f0a92aa0e8164ee0890c4bb9a039faf7)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:02.833224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158fcb2-b3ad-4040-8a8e-401b0ad14805 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:02.834871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 61/116] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:02 2022] GET /identity/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:02.835648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158fcb2-b3ad-4040-8a8e-401b0ad14805 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:02.868463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d9f42-34cd-4515-9032-f794dff5ae82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:02.869896 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d9f42-34cd-4515-9032-f794dff5ae82 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:02.870277 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d9f42-34cd-4515-9032-f794dff5ae82 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:02.870456 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d9f42-34cd-4515-9032-f794dff5ae82 None admin] PATH_INFO: `/v3/users/54524a6e8932472a80eae538a8b3a26c` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:02.889086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d9f42-34cd-4515-9032-f794dff5ae82 None admin] RBAC: Authorizing `identity:get_user(user_id=54524a6e8932472a80eae538a8b3a26c)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:02.889086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d9f42-34cd-4515-9032-f794dff5ae82 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:02.890318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 56/117] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:02 2022] GET /identity/v3/users/54524a6e8932472a80eae538a8b3a26c => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:02.892227 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d9f42-34cd-4515-9032-f794dff5ae82 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:02.941590 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158fcb2-b3ad-4040-8a8e-401b0ad14805 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:02.975941 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9af148-e996-4789-bd04-8dc6aaab5542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:02.976848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9af148-e996-4789-bd04-8dc6aaab5542 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:02.977142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9af148-e996-4789-bd04-8dc6aaab5542 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:02.977423 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9af148-e996-4789-bd04-8dc6aaab5542 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:02.979150 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9af148-e996-4789-bd04-8dc6aaab5542 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:02.980616 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9af148-e996-4789-bd04-8dc6aaab5542 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:02.982524 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 62/118] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:02.983144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9af148-e996-4789-bd04-8dc6aaab5542 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:03.001165 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d9f42-34cd-4515-9032-f794dff5ae82 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:03.018138 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:03.036436 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12479b1-ef85-4cc3-9e42-130ddfb28ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:03.037159 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12479b1-ef85-4cc3-9e42-130ddfb28ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:03.037448 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12479b1-ef85-4cc3-9e42-130ddfb28ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:03.037732 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12479b1-ef85-4cc3-9e42-130ddfb28ed6 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:03.038484 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12479b1-ef85-4cc3-9e42-130ddfb28ed6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:03.038824 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12479b1-ef85-4cc3-9e42-130ddfb28ed6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:03.040355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12479b1-ef85-4cc3-9e42-130ddfb28ed6 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:03.041720 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:03.049698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 57/119] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:03.050527 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12479b1-ef85-4cc3-9e42-130ddfb28ed6 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:03.091092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9af148-e996-4789-bd04-8dc6aaab5542 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:03.124012 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12b959-72cf-4b55-9111-321c8edec366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:03.124967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12b959-72cf-4b55-9111-321c8edec366 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:03.125254 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12b959-72cf-4b55-9111-321c8edec366 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:03.125534 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12b959-72cf-4b55-9111-321c8edec366 None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:03.126731 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa12b959-72cf-4b55-9111-321c8edec366 None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:03.128199 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa12b959-72cf-4b55-9111-321c8edec366 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:03.130343 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 63/120] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:02 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:03.131376 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e71d7-1d6e-440f-a5ad-0fdbbc28680e None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:03.131670 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e71d7-1d6e-440f-a5ad-0fdbbc28680e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:03.131952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e71d7-1d6e-440f-a5ad-0fdbbc28680e None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:03.133216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 64/121] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:03.133924 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e71d7-1d6e-440f-a5ad-0fdbbc28680e None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:03.159987 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12479b1-ef85-4cc3-9e42-130ddfb28ed6 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:03.194104 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b9316-de02-4fc5-883f-f0814ea1e1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:03.194972 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b9316-de02-4fc5-883f-f0814ea1e1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:03.195255 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b9316-de02-4fc5-883f-f0814ea1e1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:03.195533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b9316-de02-4fc5-883f-f0814ea1e1f4 None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:03.196732 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b9316-de02-4fc5-883f-f0814ea1e1f4 None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:03.198862 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b9316-de02-4fc5-883f-f0814ea1e1f4 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:03.200567 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 58/122] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:03 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:03.201356 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b9316-de02-4fc5-883f-f0814ea1e1f4 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:03.238077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e71d7-1d6e-440f-a5ad-0fdbbc28680e None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:03.239634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:03.264316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:03.275220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b7740-b175-4ad2-9668-b1ba2ecb1157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:03.276115 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b7740-b175-4ad2-9668-b1ba2ecb1157 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:03.276408 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b7740-b175-4ad2-9668-b1ba2ecb1157 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:03.276690 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b7740-b175-4ad2-9668-b1ba2ecb1157 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:03.302391 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b9316-de02-4fc5-883f-f0814ea1e1f4 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:03.337147 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d195e40-9cea-4107-ac1a-042d32bce03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:03.338390 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d195e40-9cea-4107-ac1a-042d32bce03e None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:03.338715 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d195e40-9cea-4107-ac1a-042d32bce03e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:03.339028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d195e40-9cea-4107-ac1a-042d32bce03e None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:03.339768 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d195e40-9cea-4107-ac1a-042d32bce03e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:03.341351 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d195e40-9cea-4107-ac1a-042d32bce03e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:03.344230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b7740-b175-4ad2-9668-b1ba2ecb1157 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:03.345732 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b7740-b175-4ad2-9668-b1ba2ecb1157 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:03.366733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 59/123] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Jul 12 21:52:03 2022] POST /identity/v3/projects => generated 302 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:52:03.367374 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d195e40-9cea-4107-ac1a-042d32bce03e None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:03.470770 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 65/124] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:03.471671 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b7740-b175-4ad2-9668-b1ba2ecb1157 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:03.477617 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:03.502781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d195e40-9cea-4107-ac1a-042d32bce03e None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:03.521856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:03.550441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1b655-31af-4b7b-80fb-d4fb15d7b1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:03.551857 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1b655-31af-4b7b-80fb-d4fb15d7b1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:03.552311 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1b655-31af-4b7b-80fb-d4fb15d7b1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:03.552799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1b655-31af-4b7b-80fb-d4fb15d7b1f4 None admin] PATH_INFO: `/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:03.554820 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1b655-31af-4b7b-80fb-d4fb15d7b1f4 None admin] RBAC: Authorizing `identity:get_user(user_id=f0a92aa0e8164ee0890c4bb9a039faf7)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:03.558676 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1b655-31af-4b7b-80fb-d4fb15d7b1f4 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:03.560762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 60/125] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:03 2022] GET /identity/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:03.563481 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f99f1-fab3-4c1f-8f3f-079a77d22ceb None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:03.564004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f99f1-fab3-4c1f-8f3f-079a77d22ceb None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:03.564515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f99f1-fab3-4c1f-8f3f-079a77d22ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:03.588562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-a23f99f1-fab3-4c1f-8f3f-079a77d22ceb None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:03.589827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a23f99f1-fab3-4c1f-8f3f-079a77d22ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:03.593471 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b7740-b175-4ad2-9668-b1ba2ecb1157 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:03.631480 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e9032-10b6-4e26-8cb5-958e19ff313a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:03.632374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e9032-10b6-4e26-8cb5-958e19ff313a None admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:03.632663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e9032-10b6-4e26-8cb5-958e19ff313a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:03.632976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e9032-10b6-4e26-8cb5-958e19ff313a None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5/users/a08b779b7e9045028b1b3a86939d306f/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:03.635657 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e9032-10b6-4e26-8cb5-958e19ff313a None admin] RBAC: Authorizing `identity:create_grant(project_id=73157d93d8f74bfba4cf91b3cd885fe5, user_id=a08b779b7e9045028b1b3a86939d306f, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:03.637210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e9032-10b6-4e26-8cb5-958e19ff313a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:03.651811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 66/126] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:52:03 2022] PUT /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5/users/a08b779b7e9045028b1b3a86939d306f/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:52:03.652647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e9032-10b6-4e26-8cb5-958e19ff313a None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:03.673315 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 61/127] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:03.674472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f99f1-fab3-4c1f-8f3f-079a77d22ceb None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:03.737484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:03.793972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e9032-10b6-4e26-8cb5-958e19ff313a None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:03.824181 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f99f1-fab3-4c1f-8f3f-079a77d22ceb None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:03.836078 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aacf0d-a436-4c10-b3cf-812e00e62625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:03.837355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aacf0d-a436-4c10-b3cf-812e00e62625 None admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:03.838463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aacf0d-a436-4c10-b3cf-812e00e62625 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:03.838994 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aacf0d-a436-4c10-b3cf-812e00e62625 None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5/users/f0a92aa0e8164ee0890c4bb9a039faf7/roles/160d4162340e472b9712a7f15cb2ecca` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:03.842241 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aacf0d-a436-4c10-b3cf-812e00e62625 None admin] RBAC: Authorizing `identity:create_grant(project_id=73157d93d8f74bfba4cf91b3cd885fe5, user_id=f0a92aa0e8164ee0890c4bb9a039faf7, role_id=160d4162340e472b9712a7f15cb2ecca)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:03.843758 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:03.844918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aacf0d-a436-4c10-b3cf-812e00e62625 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:03.864016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 67/128] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:52:03 2022] PUT /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5/users/f0a92aa0e8164ee0890c4bb9a039faf7/roles/160d4162340e472b9712a7f15cb2ecca => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:52:03.865240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aacf0d-a436-4c10-b3cf-812e00e62625 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:03.880324 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87297f-71aa-4980-8165-39c82c8abcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:03.881656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87297f-71aa-4980-8165-39c82c8abcff None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:03.882143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87297f-71aa-4980-8165-39c82c8abcff None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:03.883123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87297f-71aa-4980-8165-39c82c8abcff None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:03.885578 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87297f-71aa-4980-8165-39c82c8abcff None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:03.888362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87297f-71aa-4980-8165-39c82c8abcff None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:03.891748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 62/129] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:03 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:03.893163 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87297f-71aa-4980-8165-39c82c8abcff None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:04.039723 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aacf0d-a436-4c10-b3cf-812e00e62625 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:04.054652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87297f-71aa-4980-8165-39c82c8abcff None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:04.091047 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:04.116173 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08456de8-8765-4c64-a7d7-84ed5c79890e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:04.116349 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08456de8-8765-4c64-a7d7-84ed5c79890e None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:04.116797 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08456de8-8765-4c64-a7d7-84ed5c79890e None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:04.117236 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08456de8-8765-4c64-a7d7-84ed5c79890e None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:04.118710 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08456de8-8765-4c64-a7d7-84ed5c79890e None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:04.119701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0b7d8-7228-4687-82b2-fc21de1fa1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:04.120734 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0b7d8-7228-4687-82b2-fc21de1fa1ef None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:04.120887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08456de8-8765-4c64-a7d7-84ed5c79890e None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:04.121338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0b7d8-7228-4687-82b2-fc21de1fa1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:04.121847 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0b7d8-7228-4687-82b2-fc21de1fa1ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:04.123304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 68/130] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:03 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:04.126913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08456de8-8765-4c64-a7d7-84ed5c79890e None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:04.141665 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0b7d8-7228-4687-82b2-fc21de1fa1ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:04.154664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-96f0b7d8-7228-4687-82b2-fc21de1fa1ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:04.154664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:04.154664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:04.154664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:04.154664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:04.154664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:04.154664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:04.154664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:04.154664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:04.154664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:04.154664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:04.154664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:04.154664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:04.154664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:04.154664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:04.154664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:04.155586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:04.155586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:04.155586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:04.155586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:04.155586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:04.155586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:04.155586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:04.155586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:04.155586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:04.155586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:04.155586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:04.155586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:04.155586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:04.155586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:04.155586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:04.155586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:04.156350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:04.156350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:04.156350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:04.156350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:04.156350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:04.156350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:04.156350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:04.156350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:04.156350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:04.156350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:04.156350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:04.156350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:04.158278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 63/131] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:04.161500 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0b7d8-7228-4687-82b2-fc21de1fa1ef None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:04.233181 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:04.255223 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08456de8-8765-4c64-a7d7-84ed5c79890e None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:04.294626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2da5de-95ce-4d69-828b-94f6f2917096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:04.295660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2da5de-95ce-4d69-828b-94f6f2917096 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:04.296173 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2da5de-95ce-4d69-828b-94f6f2917096 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:04.296653 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2da5de-95ce-4d69-828b-94f6f2917096 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:04.297723 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2da5de-95ce-4d69-828b-94f6f2917096 None admin] RBAC: Adding query filter params (role.id=d43f8f61eb5f4489acfd5fba99b85919, scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5, user.id=54524a6e8932472a80eae538a8b3a26c) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:04.298238 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2da5de-95ce-4d69-828b-94f6f2917096 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:04.299916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2da5de-95ce-4d69-828b-94f6f2917096 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:04.308083 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 69/132] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:04 2022] GET /identity/v3/role_assignments?user.id=54524a6e8932472a80eae538a8b3a26c&scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5&role.id=d43f8f61eb5f4489acfd5fba99b85919 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:04.309514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2da5de-95ce-4d69-828b-94f6f2917096 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:04.361301 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0b7d8-7228-4687-82b2-fc21de1fa1ef None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:04.423021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a4b81-a9cb-4071-8586-e1a03d48c308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:04.423991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:04.425199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a4b81-a9cb-4071-8586-e1a03d48c308 None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:04.426550 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a4b81-a9cb-4071-8586-e1a03d48c308 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:04.427884 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a4b81-a9cb-4071-8586-e1a03d48c308 None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5/users/f0a92aa0e8164ee0890c4bb9a039faf7/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:04.433985 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a4b81-a9cb-4071-8586-e1a03d48c308 None admin] RBAC: Authorizing `identity:create_grant(project_id=73157d93d8f74bfba4cf91b3cd885fe5, user_id=f0a92aa0e8164ee0890c4bb9a039faf7, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:04.436663 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a4b81-a9cb-4071-8586-e1a03d48c308 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:04.467095 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 64/133] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:52:04 2022] PUT /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5/users/f0a92aa0e8164ee0890c4bb9a039faf7/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:52:04.521210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2da5de-95ce-4d69-828b-94f6f2917096 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:04.577318 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7142a1-ac69-4090-9bfe-b993a5f9a667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:04.578365 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7142a1-ac69-4090-9bfe-b993a5f9a667 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:04.578858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7142a1-ac69-4090-9bfe-b993a5f9a667 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:04.579370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7142a1-ac69-4090-9bfe-b993a5f9a667 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:04.580364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7142a1-ac69-4090-9bfe-b993a5f9a667 None admin] RBAC: Adding query filter params (name=member) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:04.580931 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7142a1-ac69-4090-9bfe-b993a5f9a667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:04.582649 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7142a1-ac69-4090-9bfe-b993a5f9a667 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:04.597099 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 70/134] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:04.604247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a4b81-a9cb-4071-8586-e1a03d48c308 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:04.745785 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a4b81-a9cb-4071-8586-e1a03d48c308 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:04.747856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:04.785669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb61dd-6a0b-42b9-851a-8ef2b9560e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:04.786567 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb61dd-6a0b-42b9-851a-8ef2b9560e15 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:04.786853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb61dd-6a0b-42b9-851a-8ef2b9560e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:04.787135 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb61dd-6a0b-42b9-851a-8ef2b9560e15 None admin] PATH_INFO: `/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:04.789021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb61dd-6a0b-42b9-851a-8ef2b9560e15 None admin] RBAC: Authorizing `identity:get_user(user_id=f0a92aa0e8164ee0890c4bb9a039faf7)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:04.790498 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb61dd-6a0b-42b9-851a-8ef2b9560e15 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:04.792688 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 65/135] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:04 2022] GET /identity/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:04.798245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7142a1-ac69-4090-9bfe-b993a5f9a667 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:04.919509 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7142a1-ac69-4090-9bfe-b993a5f9a667 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:04.936778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:04.966989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6cefc-42d2-4a64-8a30-1d61eda97af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:04.967886 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6cefc-42d2-4a64-8a30-1d61eda97af5 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:04.968182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6cefc-42d2-4a64-8a30-1d61eda97af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:04.968464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6cefc-42d2-4a64-8a30-1d61eda97af5 None admin] PATH_INFO: `/v3/projects/1693b44e5fea4067af432358e61ee9f5` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:04.973471 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6cefc-42d2-4a64-8a30-1d61eda97af5 None admin] RBAC: Authorizing `identity:get_project(project_id=1693b44e5fea4067af432358e61ee9f5)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:04.974987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6cefc-42d2-4a64-8a30-1d61eda97af5 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:04.976875 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 71/136] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:04 2022] GET /identity/v3/projects/1693b44e5fea4067af432358e61ee9f5 => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:04.983263 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb61dd-6a0b-42b9-851a-8ef2b9560e15 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:05.097495 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb61dd-6a0b-42b9-851a-8ef2b9560e15 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:05.133727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5705e0e-fe62-4a65-a3fc-4da19e577cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:05.134728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5705e0e-fe62-4a65-a3fc-4da19e577cff None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:05.135016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5705e0e-fe62-4a65-a3fc-4da19e577cff None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:05.135302 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5705e0e-fe62-4a65-a3fc-4da19e577cff None admin] PATH_INFO: `/v3/projects/1693b44e5fea4067af432358e61ee9f5/users/f0a92aa0e8164ee0890c4bb9a039faf7/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:05.141790 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5705e0e-fe62-4a65-a3fc-4da19e577cff None admin] RBAC: Authorizing `identity:create_grant(project_id=1693b44e5fea4067af432358e61ee9f5, user_id=f0a92aa0e8164ee0890c4bb9a039faf7, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:05.143375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5705e0e-fe62-4a65-a3fc-4da19e577cff None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:05.160703 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 66/137] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:52:04 2022] PUT /identity/v3/projects/1693b44e5fea4067af432358e61ee9f5/users/f0a92aa0e8164ee0890c4bb9a039faf7/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:52:06.002388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccafd8b-ec29-4c65-8da9-8b9588e7d334 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:06.002903 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccafd8b-ec29-4c65-8da9-8b9588e7d334 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:06.002977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccafd8b-ec29-4c65-8da9-8b9588e7d334 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:06.004508 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 72/138] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:06.010702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a57c65-8aaa-498d-81c9-8b1f07a6c489 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:06.011446 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a57c65-8aaa-498d-81c9-8b1f07a6c489 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:06.011446 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a57c65-8aaa-498d-81c9-8b1f07a6c489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:06.031455 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-c8a57c65-8aaa-498d-81c9-8b1f07a6c489 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:06.032656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c8a57c65-8aaa-498d-81c9-8b1f07a6c489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:06.059493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:06.122247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 67/139] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:06.148117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccafd8b-ec29-4c65-8da9-8b9588e7d334 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:06.278265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccafd8b-ec29-4c65-8da9-8b9588e7d334 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:06.314192 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4574ae0-dcf2-46cf-bd70-187634deaa2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:06.315000 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4574ae0-dcf2-46cf-bd70-187634deaa2c None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:06.315289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4574ae0-dcf2-46cf-bd70-187634deaa2c None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:06.315571 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4574ae0-dcf2-46cf-bd70-187634deaa2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:06.317215 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4574ae0-dcf2-46cf-bd70-187634deaa2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:06.318698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4574ae0-dcf2-46cf-bd70-187634deaa2c None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:06.320641 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 73/140] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:06.330016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a57c65-8aaa-498d-81c9-8b1f07a6c489 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:06.454294 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a57c65-8aaa-498d-81c9-8b1f07a6c489 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:06.499271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9287d-c825-4c85-a27f-e246ecd3b720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:06.499964 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9287d-c825-4c85-a27f-e246ecd3b720 None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:06.500253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9287d-c825-4c85-a27f-e246ecd3b720 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:06.500536 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9287d-c825-4c85-a27f-e246ecd3b720 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:06.501294 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9287d-c825-4c85-a27f-e246ecd3b720 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:06.503160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9287d-c825-4c85-a27f-e246ecd3b720 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:06.536204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 68/141] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:52:06 2022] POST /identity/v3/users => generated 283 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:52:06.767435 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019899dc-74a0-46db-b4cc-e1697863fb39 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:06.767821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019899dc-74a0-46db-b4cc-e1697863fb39 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:06.768052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019899dc-74a0-46db-b4cc-e1697863fb39 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:06.769763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 74/142] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:06.774828 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559af6a-bef7-4e9e-99af-0393e4d0ebd6 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:06.775202 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559af6a-bef7-4e9e-99af-0393e4d0ebd6 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:06.775533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559af6a-bef7-4e9e-99af-0393e4d0ebd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:06.798182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-8559af6a-bef7-4e9e-99af-0393e4d0ebd6 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:06.799341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8559af6a-bef7-4e9e-99af-0393e4d0ebd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:06.816418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:06.887795 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 69/143] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:06.898345 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf000c2-f7de-478e-90e0-c6accdc13f0b None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:06.898638 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf000c2-f7de-478e-90e0-c6accdc13f0b None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:06.898933 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf000c2-f7de-478e-90e0-c6accdc13f0b None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:06.900623 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 75/144] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:06.906544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364ed0d-7524-4930-be32-5327338c14d9 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:06.906843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364ed0d-7524-4930-be32-5327338c14d9 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:06.907131 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364ed0d-7524-4930-be32-5327338c14d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:06.926686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-a364ed0d-7524-4930-be32-5327338c14d9 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:06.927768 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf000c2-f7de-478e-90e0-c6accdc13f0b None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:06.927996 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a364ed0d-7524-4930-be32-5327338c14d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:07.046034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 70/145] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:07.057502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:07.077160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364ed0d-7524-4930-be32-5327338c14d9 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:07.161014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf000c2-f7de-478e-90e0-c6accdc13f0b None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:07.209620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:07.240995 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364ed0d-7524-4930-be32-5327338c14d9 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:07.250484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba273d69-3d73-405f-9781-8bc0b4480d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:07.251541 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba273d69-3d73-405f-9781-8bc0b4480d41 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:07.251982 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba273d69-3d73-405f-9781-8bc0b4480d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:07.252451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba273d69-3d73-405f-9781-8bc0b4480d41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:07.281445 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba273d69-3d73-405f-9781-8bc0b4480d41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:07.284637 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180700f5-0808-42a3-83d3-66a9fa8bf5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:07.285592 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180700f5-0808-42a3-83d3-66a9fa8bf5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:07.286062 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180700f5-0808-42a3-83d3-66a9fa8bf5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:07.286495 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180700f5-0808-42a3-83d3-66a9fa8bf5d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:07.295370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-ba273d69-3d73-405f-9781-8bc0b4480d41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:52:07.295370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:07.295370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:07.295370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:07.295370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:07.295370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:07.295370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:07.295370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:07.295370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:07.295370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:07.295370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:07.295370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:07.295370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:07.295370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:07.295370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:07.295370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:07.296448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:07.296448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:07.296448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:07.296448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:07.296448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:07.296448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:07.296448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:07.296448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:07.296448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:07.296448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:07.296448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:07.296448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:07.296448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:07.296448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:07.296448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:07.296448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:07.297291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:07.297291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:07.297291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:07.297291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:07.297291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:07.297291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:07.297291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:07.297291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:07.297291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:07.297291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:07.297291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:52:07.297291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:07.299622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 76/146] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:52:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:52:07.300639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180700f5-0808-42a3-83d3-66a9fa8bf5d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:07.313552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-180700f5-0808-42a3-83d3-66a9fa8bf5d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:07.313552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:07.313552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:07.313552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:07.313552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:07.313552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:07.313552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:07.313552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:07.313552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:07.313552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:07.313552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:07.313552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:07.313552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:07.313552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:07.313552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:07.313552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:07.314447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:07.314447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:07.314447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:07.314447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:07.314447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:07.314447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:07.314447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:07.314447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:07.314447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:07.314447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:07.314447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:07.314447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:07.314447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:07.314447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:07.314447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:07.314447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:07.315237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:07.315237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:07.315237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:07.315237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:07.315237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:07.315237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:07.315237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:07.315237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:07.315237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:07.315237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:07.315237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:07.315237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:07.318236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 71/147] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:52:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:07.324354 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba273d69-3d73-405f-9781-8bc0b4480d41 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:07.324991 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180700f5-0808-42a3-83d3-66a9fa8bf5d0 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:07.455779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180700f5-0808-42a3-83d3-66a9fa8bf5d0 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:07.484323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba273d69-3d73-405f-9781-8bc0b4480d41 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:07.500285 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4857d-4cd3-494e-bee1-a91ad7556a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:07.501328 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4857d-4cd3-494e-bee1-a91ad7556a2b None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:07.501812 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4857d-4cd3-494e-bee1-a91ad7556a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:07.502305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4857d-4cd3-494e-bee1-a91ad7556a2b None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:07.503293 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be4857d-4cd3-494e-bee1-a91ad7556a2b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:07.503766 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be4857d-4cd3-494e-bee1-a91ad7556a2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:07.505461 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be4857d-4cd3-494e-bee1-a91ad7556a2b None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:07.510749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:07.512924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:07.519388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 72/148] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:52:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:07.522864 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4857d-4cd3-494e-bee1-a91ad7556a2b None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:07.531962 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ed973-0e38-4b63-8bf3-c27e3ecda178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:07.532758 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ed973-0e38-4b63-8bf3-c27e3ecda178 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:07.533124 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ed973-0e38-4b63-8bf3-c27e3ecda178 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:07.533475 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ed973-0e38-4b63-8bf3-c27e3ecda178 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:07.534849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ed973-0e38-4b63-8bf3-c27e3ecda178 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:07.535240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ed973-0e38-4b63-8bf3-c27e3ecda178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:07.537129 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ed973-0e38-4b63-8bf3-c27e3ecda178 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:07.552741 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 77/149] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:07.558065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ed973-0e38-4b63-8bf3-c27e3ecda178 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:07.703090 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4857d-4cd3-494e-bee1-a91ad7556a2b None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:07.742112 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d65380-2198-437e-93a1-bf88784a089b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:07.743529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d65380-2198-437e-93a1-bf88784a089b None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:07.744498 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d65380-2198-437e-93a1-bf88784a089b None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:07.745029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d65380-2198-437e-93a1-bf88784a089b None admin] PATH_INFO: `/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:07.746523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d65380-2198-437e-93a1-bf88784a089b None admin] RBAC: Authorizing `identity:get_user(user_id=f0a92aa0e8164ee0890c4bb9a039faf7)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:07.748222 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d65380-2198-437e-93a1-bf88784a089b None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:07.751675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 73/150] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:07 2022] GET /identity/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7 => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:07.753314 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78e28b-b154-48d7-bd7e-3e3d5a0959a8 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:07.753813 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78e28b-b154-48d7-bd7e-3e3d5a0959a8 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:07.754475 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78e28b-b154-48d7-bd7e-3e3d5a0959a8 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:07.756590 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 74/151] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:07.762913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78e28b-b154-48d7-bd7e-3e3d5a0959a8 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:07.763797 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ed973-0e38-4b63-8bf3-c27e3ecda178 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:07.849126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:07.852921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237a558-4acd-4a72-b197-382930d94ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:07.854451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237a558-4acd-4a72-b197-382930d94ffe None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:07.854781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237a558-4acd-4a72-b197-382930d94ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:07.855056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237a558-4acd-4a72-b197-382930d94ffe None admin] PATH_INFO: `/v3/users/a08b779b7e9045028b1b3a86939d306f` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:07.861549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237a558-4acd-4a72-b197-382930d94ffe None admin] RBAC: Authorizing `identity:get_user(user_id=a08b779b7e9045028b1b3a86939d306f)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:07.863058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237a558-4acd-4a72-b197-382930d94ffe None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:07.865183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 78/152] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:07 2022] GET /identity/v3/users/a08b779b7e9045028b1b3a86939d306f => generated 247 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:07.866516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071a86b-46f4-4ae6-906f-9f1ceb71ce4f None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:07.866812 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071a86b-46f4-4ae6-906f-9f1ceb71ce4f None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:07.867099 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071a86b-46f4-4ae6-906f-9f1ceb71ce4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:07.894331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-1071a86b-46f4-4ae6-906f-9f1ceb71ce4f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:07.895550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-1071a86b-46f4-4ae6-906f-9f1ceb71ce4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:07.917304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78e28b-b154-48d7-bd7e-3e3d5a0959a8 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:07.942398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:07.996033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419be4a-b79b-4673-9463-d63980f61b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:07.996737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419be4a-b79b-4673-9463-d63980f61b98 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:07.997017 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419be4a-b79b-4673-9463-d63980f61b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:07.997296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419be4a-b79b-4673-9463-d63980f61b98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:08.018266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 79/153] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:08.019024 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071a86b-46f4-4ae6-906f-9f1ceb71ce4f None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:08.125891 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419be4a-b79b-4673-9463-d63980f61b98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:08.128328 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419be4a-b79b-4673-9463-d63980f61b98 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:08.235364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071a86b-46f4-4ae6-906f-9f1ceb71ce4f None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:08.243955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:08.296901 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca19b3e-c88d-44c2-8f73-113b32b3e822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:08.297574 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca19b3e-c88d-44c2-8f73-113b32b3e822 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:08.297863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca19b3e-c88d-44c2-8f73-113b32b3e822 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:08.298144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca19b3e-c88d-44c2-8f73-113b32b3e822 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:08.359334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 75/154] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 21:52:08.360068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419be4a-b79b-4673-9463-d63980f61b98 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:08.377150 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:08.414684 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca19b3e-c88d-44c2-8f73-113b32b3e822 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:08.416762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca19b3e-c88d-44c2-8f73-113b32b3e822 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:08.496540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419be4a-b79b-4673-9463-d63980f61b98 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:08.537742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82e391-abd9-445f-b2ec-89399efdf875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:08.539919 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82e391-abd9-445f-b2ec-89399efdf875 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:08.541154 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82e391-abd9-445f-b2ec-89399efdf875 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:08.542414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82e391-abd9-445f-b2ec-89399efdf875 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:08.566459 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e82e391-abd9-445f-b2ec-89399efdf875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:08.581336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-6e82e391-abd9-445f-b2ec-89399efdf875 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:08.581336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:08.581336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:08.581336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:08.581336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:08.581336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:08.581336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:08.581336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:08.581336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:08.581336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:08.581336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:08.581336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:08.581336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:08.581336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:08.581336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:08.581336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:08.582272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:08.582272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:08.582272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:08.582272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:08.582272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:08.582272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:08.582272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:08.582272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:08.582272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:08.582272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:08.582272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:08.582272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:08.582272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:08.582272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:08.582272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:08.582272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:08.583346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:08.583346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:08.583346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:08.583346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:08.583346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:08.583346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:08.583346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:08.583346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:08.583346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:08.583346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:08.583346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:08.583346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:08.587551 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 76/155] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:08.595106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f181e-ba54-42b6-8eee-0c4175bea192 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:08.596570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f181e-ba54-42b6-8eee-0c4175bea192 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:08.599704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f181e-ba54-42b6-8eee-0c4175bea192 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:08.601519 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 77/156] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:08 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:08.602644 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f181e-ba54-42b6-8eee-0c4175bea192 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:08.624648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 80/157] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:08.626021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdab01a-02a0-49cc-8538-c15910141dd4 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:08.626319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdab01a-02a0-49cc-8538-c15910141dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:08.626603 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdab01a-02a0-49cc-8538-c15910141dd4 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:08.628043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 81/158] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:08.633155 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdab01a-02a0-49cc-8538-c15910141dd4 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:08.648703 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:08.790302 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f181e-ba54-42b6-8eee-0c4175bea192 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:08.791331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:08.856867 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4bfa7d-40ab-416a-a6ac-cfeb96112c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:08.857749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4bfa7d-40ab-416a-a6ac-cfeb96112c54 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:08.858119 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4bfa7d-40ab-416a-a6ac-cfeb96112c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:08.858377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4bfa7d-40ab-416a-a6ac-cfeb96112c54 None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:08.859775 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4bfa7d-40ab-416a-a6ac-cfeb96112c54 None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:08.861339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4bfa7d-40ab-416a-a6ac-cfeb96112c54 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:08.862126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdab01a-02a0-49cc-8538-c15910141dd4 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:08.863065 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 78/159] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:08 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:08.864548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8ea07-673b-4acc-be12-85f3fe602573 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:08.864865 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8ea07-673b-4acc-be12-85f3fe602573 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:08.865149 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8ea07-673b-4acc-be12-85f3fe602573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:08.887706 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-4ba8ea07-673b-4acc-be12-85f3fe602573 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:08.889180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4ba8ea07-673b-4acc-be12-85f3fe602573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:08.934509 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01866c1-4abf-404b-ad73-d4517e72faf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:08.935246 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01866c1-4abf-404b-ad73-d4517e72faf6 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:08.935532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01866c1-4abf-404b-ad73-d4517e72faf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:08.935807 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01866c1-4abf-404b-ad73-d4517e72faf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:08.936614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01866c1-4abf-404b-ad73-d4517e72faf6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:08.938219 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01866c1-4abf-404b-ad73-d4517e72faf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:08.941419 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01866c1-4abf-404b-ad73-d4517e72faf6 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:08.964105 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 82/160] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:08.968750 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01866c1-4abf-404b-ad73-d4517e72faf6 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:09.030920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 79/161] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:09.032504 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382d136-2c67-4863-879b-68424e0d6fa2 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:09.033487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382d136-2c67-4863-879b-68424e0d6fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:09.033780 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382d136-2c67-4863-879b-68424e0d6fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:09.062035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-5382d136-2c67-4863-879b-68424e0d6fa2 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:09.063240 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-5382d136-2c67-4863-879b-68424e0d6fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:09.089145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:09.176599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01866c1-4abf-404b-ad73-d4517e72faf6 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:09.180379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 80/162] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:09.182817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382d136-2c67-4863-879b-68424e0d6fa2 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:09.258497 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5ba28-7f28-46b4-89a4-eb1242a7eb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:09.259921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5ba28-7f28-46b4-89a4-eb1242a7eb0b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:09.260205 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5ba28-7f28-46b4-89a4-eb1242a7eb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:09.260579 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5ba28-7f28-46b4-89a4-eb1242a7eb0b None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:09.261844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5ba28-7f28-46b4-89a4-eb1242a7eb0b None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:09.263538 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5ba28-7f28-46b4-89a4-eb1242a7eb0b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:09.265406 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 83/163] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:08 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:09.266247 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5ba28-7f28-46b4-89a4-eb1242a7eb0b None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:09.273264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:09.362051 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382d136-2c67-4863-879b-68424e0d6fa2 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:09.388485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5ba28-7f28-46b4-89a4-eb1242a7eb0b None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:09.418670 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00574c56-3de5-4c04-9af8-a662917bce5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:09.418890 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00574c56-3de5-4c04-9af8-a662917bce5a None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:09.419129 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00574c56-3de5-4c04-9af8-a662917bce5a None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:09.419287 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00574c56-3de5-4c04-9af8-a662917bce5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:09.420127 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00574c56-3de5-4c04-9af8-a662917bce5a None admin] RBAC: Adding query filter params (role.id=160d4162340e472b9712a7f15cb2ecca, scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5, user.id=f0a92aa0e8164ee0890c4bb9a039faf7) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:09.420435 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00574c56-3de5-4c04-9af8-a662917bce5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:09.421936 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00574c56-3de5-4c04-9af8-a662917bce5a None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:09.444766 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 81/164] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:09 2022] GET /identity/v3/role_assignments?user.id=f0a92aa0e8164ee0890c4bb9a039faf7&scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5&role.id=160d4162340e472b9712a7f15cb2ecca => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:09.444766 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00574c56-3de5-4c04-9af8-a662917bce5a None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:09.489219 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4ba94-61ff-43c3-9132-95842a57c200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:09.490109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4ba94-61ff-43c3-9132-95842a57c200 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:09.490389 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4ba94-61ff-43c3-9132-95842a57c200 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:09.490663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4ba94-61ff-43c3-9132-95842a57c200 None admin] PATH_INFO: `/v3/users/54524a6e8932472a80eae538a8b3a26c` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:09.494997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4ba94-61ff-43c3-9132-95842a57c200 None admin] RBAC: Authorizing `identity:get_user(user_id=54524a6e8932472a80eae538a8b3a26c)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:09.504738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4ba94-61ff-43c3-9132-95842a57c200 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:09.504738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 84/165] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:09 2022] GET /identity/v3/users/54524a6e8932472a80eae538a8b3a26c => generated 289 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:09.504738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4ba94-61ff-43c3-9132-95842a57c200 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:09.541431 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:09.757119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:09.831053 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00574c56-3de5-4c04-9af8-a662917bce5a None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:09.846291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4ba94-61ff-43c3-9132-95842a57c200 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:09.914281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ef529-e1b8-4dee-bb16-636c20796a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:09.915309 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ef529-e1b8-4dee-bb16-636c20796a02 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:09.916152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ef529-e1b8-4dee-bb16-636c20796a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:09.920746 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ef529-e1b8-4dee-bb16-636c20796a02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:09.944110 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ef529-e1b8-4dee-bb16-636c20796a02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:09.952613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d65b1-41ef-4716-86ef-b061cde4bf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:09.953454 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d65b1-41ef-4716-86ef-b061cde4bf73 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:09.956885 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d65b1-41ef-4716-86ef-b061cde4bf73 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:09.956885 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d65b1-41ef-4716-86ef-b061cde4bf73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:09.957091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-0d7ef529-e1b8-4dee-bb16-636c20796a02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:09.957091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:09.957091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:09.957091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:09.957091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:09.957091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:09.957091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:09.957091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:09.957091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:09.957091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:09.957091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:09.957091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:09.957091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:09.957091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:09.957091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:09.957091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:09.957906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:09.957906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:09.957906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:09.957906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:09.957906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:09.957906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:09.957906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:09.957906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:09.957906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:09.957906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:09.957906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:09.957906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:09.957906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:09.957906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:09.957906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:09.957906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:09.958914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:09.958914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:09.958914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:09.958914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:09.958914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:09.958914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:09.958914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:09.958914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:09.958914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:09.958914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:09.958914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:09.958914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:09.958914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 82/166] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 515 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:09.958914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ef529-e1b8-4dee-bb16-636c20796a02 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:10.008816 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d65b1-41ef-4716-86ef-b061cde4bf73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:10.015374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-040d65b1-41ef-4716-86ef-b061cde4bf73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:10.015374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:10.015374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:10.015374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:10.015374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:10.015374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:10.015374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:10.015374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:10.015374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:10.015374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:10.015374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:10.015374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:10.015374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:10.015374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:10.015374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:10.015374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:10.016285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:10.016285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:10.016285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:10.016285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:10.016285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:10.016285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:10.016285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:10.016285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:10.016285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:10.016285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:10.016285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:10.016285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:10.016285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:10.016285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:10.016285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:10.016285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:10.017111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:10.017111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:10.017111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:10.017111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:10.017111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:10.017111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:10.017111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:10.017111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:10.017111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:10.017111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:10.017111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:10.017111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:10.017111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 85/167] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 516 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:10.018471 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d65b1-41ef-4716-86ef-b061cde4bf73 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:10.137869 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:10.174233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ef529-e1b8-4dee-bb16-636c20796a02 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:10.215313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d65b1-41ef-4716-86ef-b061cde4bf73 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:10.217197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549bffe-67c4-41fb-b805-44b53266f8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:10.218836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549bffe-67c4-41fb-b805-44b53266f8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:10.219125 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549bffe-67c4-41fb-b805-44b53266f8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:10.219401 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549bffe-67c4-41fb-b805-44b53266f8e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:10.220257 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549bffe-67c4-41fb-b805-44b53266f8e4 None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5, user.id=a08b779b7e9045028b1b3a86939d306f) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:10.220568 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549bffe-67c4-41fb-b805-44b53266f8e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:10.222060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549bffe-67c4-41fb-b805-44b53266f8e4 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:10.230011 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 83/168] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:09 2022] GET /identity/v3/role_assignments?user.id=a08b779b7e9045028b1b3a86939d306f&scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5&role.id=053d1730ed764973abb1490894e07681 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:10.233165 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549bffe-67c4-41fb-b805-44b53266f8e4 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:10.266551 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:10.279595 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b0c17-edef-4fe2-bdc9-d15e04564437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:10.280473 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b0c17-edef-4fe2-bdc9-d15e04564437 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:10.280767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b0c17-edef-4fe2-bdc9-d15e04564437 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:10.281041 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b0c17-edef-4fe2-bdc9-d15e04564437 None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:10.282302 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b0c17-edef-4fe2-bdc9-d15e04564437 None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:10.283841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b0c17-edef-4fe2-bdc9-d15e04564437 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:10.285617 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 86/169] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:10 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:10.286805 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e92ec-c3e2-4093-a8c8-6f700447b57b None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:10.287085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e92ec-c3e2-4093-a8c8-6f700447b57b None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:10.287359 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e92ec-c3e2-4093-a8c8-6f700447b57b None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:10.297886 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 87/170] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:10 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:10.298756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e92ec-c3e2-4093-a8c8-6f700447b57b None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:10.376618 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549bffe-67c4-41fb-b805-44b53266f8e4 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:10.417486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaecc22-6b51-40ff-9df6-37476c4dc7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:10.418184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaecc22-6b51-40ff-9df6-37476c4dc7c0 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:10.418468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaecc22-6b51-40ff-9df6-37476c4dc7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:10.418750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaecc22-6b51-40ff-9df6-37476c4dc7c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:10.419524 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaecc22-6b51-40ff-9df6-37476c4dc7c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:10.419833 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaecc22-6b51-40ff-9df6-37476c4dc7c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:10.421322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaecc22-6b51-40ff-9df6-37476c4dc7c0 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:10.430568 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 84/171] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:10.436764 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584a406-6fdc-43e2-bf8b-69448ceba8ae None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:10.437443 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584a406-6fdc-43e2-bf8b-69448ceba8ae None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:10.437738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584a406-6fdc-43e2-bf8b-69448ceba8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:10.445041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:10.459693 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-f584a406-6fdc-43e2-bf8b-69448ceba8ae None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:10.461117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-f584a406-6fdc-43e2-bf8b-69448ceba8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:10.482969 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e92ec-c3e2-4093-a8c8-6f700447b57b None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:10.528322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681239b-0a9a-4341-ab89-c7e22bd3ac7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:10.528962 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681239b-0a9a-4341-ab89-c7e22bd3ac7b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:10.529248 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681239b-0a9a-4341-ab89-c7e22bd3ac7b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:10.529549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681239b-0a9a-4341-ab89-c7e22bd3ac7b None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:10.530415 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681239b-0a9a-4341-ab89-c7e22bd3ac7b None admin] RBAC: Adding query filter params (name=member) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:10.530768 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681239b-0a9a-4341-ab89-c7e22bd3ac7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:10.532099 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681239b-0a9a-4341-ab89-c7e22bd3ac7b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:10.542067 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 88/172] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:10.542901 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681239b-0a9a-4341-ab89-c7e22bd3ac7b None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:10.555127 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 85/173] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:10.556690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9fb0d-4748-467f-91cc-1db84369971e None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:10.557126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9fb0d-4748-467f-91cc-1db84369971e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:10.557410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9fb0d-4748-467f-91cc-1db84369971e None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:10.558755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 86/174] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:10.559486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9fb0d-4748-467f-91cc-1db84369971e None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:10.609880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:10.651935 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681239b-0a9a-4341-ab89-c7e22bd3ac7b None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:10.667831 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9fb0d-4748-467f-91cc-1db84369971e None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:10.687722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0001a2-05b5-432b-ac19-6b00451697ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:10.688613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0001a2-05b5-432b-ac19-6b00451697ce None admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:10.688932 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0001a2-05b5-432b-ac19-6b00451697ce None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:10.689213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0001a2-05b5-432b-ac19-6b00451697ce None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5/users/54524a6e8932472a80eae538a8b3a26c/roles/d43f8f61eb5f4489acfd5fba99b85919` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:10.691424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0001a2-05b5-432b-ac19-6b00451697ce None admin] RBAC: Authorizing `identity:create_grant(project_id=73157d93d8f74bfba4cf91b3cd885fe5, user_id=54524a6e8932472a80eae538a8b3a26c, role_id=d43f8f61eb5f4489acfd5fba99b85919)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:10.692938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0001a2-05b5-432b-ac19-6b00451697ce None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:10.705066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462969c3-70f6-4c97-a5dd-1dc0a81c5544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:10.705942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462969c3-70f6-4c97-a5dd-1dc0a81c5544 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:10.706203 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 89/175] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:52:10 2022] PUT /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5/users/54524a6e8932472a80eae538a8b3a26c/roles/d43f8f61eb5f4489acfd5fba99b85919 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:52:10.706302 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462969c3-70f6-4c97-a5dd-1dc0a81c5544 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:10.706490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462969c3-70f6-4c97-a5dd-1dc0a81c5544 None admin] PATH_INFO: `/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:10.707058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0001a2-05b5-432b-ac19-6b00451697ce None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:10.707770 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462969c3-70f6-4c97-a5dd-1dc0a81c5544 None admin] RBAC: Authorizing `identity:get_user(user_id=f0a92aa0e8164ee0890c4bb9a039faf7)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:10.709305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462969c3-70f6-4c97-a5dd-1dc0a81c5544 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:10.711212 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 87/176] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:10 2022] GET /identity/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:10.712448 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315ff1f-2c9a-4f7d-a3d4-67db7e964337 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:10.712795 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315ff1f-2c9a-4f7d-a3d4-67db7e964337 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:10.713116 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315ff1f-2c9a-4f7d-a3d4-67db7e964337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:10.718396 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:10.731784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-1315ff1f-2c9a-4f7d-a3d4-67db7e964337 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:10.733071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1315ff1f-2c9a-4f7d-a3d4-67db7e964337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:10.812793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 88/177] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:10.814196 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbd826-5335-44ea-907a-d093fdf5e279 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:10.814490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbd826-5335-44ea-907a-d093fdf5e279 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:10.814766 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbd826-5335-44ea-907a-d093fdf5e279 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:10.816521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 89/178] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:10.817133 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbd826-5335-44ea-907a-d093fdf5e279 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:10.849531 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0001a2-05b5-432b-ac19-6b00451697ce None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:10.874085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:10.892105 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8d69c-d755-4f85-b10d-3af8b6d317ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:10.893374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8d69c-d755-4f85-b10d-3af8b6d317ae None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:10.893836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8d69c-d755-4f85-b10d-3af8b6d317ae None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:10.894309 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8d69c-d755-4f85-b10d-3af8b6d317ae None admin] PATH_INFO: `/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:10.895726 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8d69c-d755-4f85-b10d-3af8b6d317ae None admin] RBAC: Authorizing `identity:get_user(user_id=f0a92aa0e8164ee0890c4bb9a039faf7)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:10.897712 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8d69c-d755-4f85-b10d-3af8b6d317ae None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:10.900418 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 90/179] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:10 2022] GET /identity/v3/users/f0a92aa0e8164ee0890c4bb9a039faf7 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:10.901430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8d69c-d755-4f85-b10d-3af8b6d317ae None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:10.952450 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbd826-5335-44ea-907a-d093fdf5e279 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:10.991354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:10.998724 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c8b13-88b5-48f3-91c5-9ebf64cec0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:10.999126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c8b13-88b5-48f3-91c5-9ebf64cec0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:10.999417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c8b13-88b5-48f3-91c5-9ebf64cec0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:10.999704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c8b13-88b5-48f3-91c5-9ebf64cec0e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:11.012540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c8b13-88b5-48f3-91c5-9ebf64cec0e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:11.015189 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8d69c-d755-4f85-b10d-3af8b6d317ae None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:11.024761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-ec6c8b13-88b5-48f3-91c5-9ebf64cec0e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:52:11.024761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:11.024761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:11.024761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:11.024761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:11.024761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:11.024761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:11.024761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:11.024761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:11.024761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:11.024761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:11.024761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:11.024761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:11.024761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:11.024761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:11.024761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:11.026418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:11.026418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:11.026418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:11.026418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:11.026418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:11.026418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:11.026418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:11.026418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:11.026418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:11.026418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:11.026418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:11.026418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:11.026418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:11.026418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:11.026418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:11.026418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:11.027242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:11.027242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:11.027242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:11.027242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:11.027242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:11.027242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:11.027242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:11.027242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:11.027242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:11.027242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:11.027242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:52:11.027242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:11.027242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 90/180] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:52:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:11.032757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a701cc-5075-4860-870b-fe57f0321914 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:11.032757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a701cc-5075-4860-870b-fe57f0321914 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:11.032757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a701cc-5075-4860-870b-fe57f0321914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:11.053783 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-18a701cc-5075-4860-870b-fe57f0321914 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:11.055211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-18a701cc-5075-4860-870b-fe57f0321914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:11.059136 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1789dd5-1d9b-475f-8b10-91b0214e3586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:11.059587 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1789dd5-1d9b-475f-8b10-91b0214e3586 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:11.059873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1789dd5-1d9b-475f-8b10-91b0214e3586 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:11.060151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1789dd5-1d9b-475f-8b10-91b0214e3586 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:11.123892 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:11.125882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 91/181] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:11.126755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a701cc-5075-4860-870b-fe57f0321914 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:11.146298 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1789dd5-1d9b-475f-8b10-91b0214e3586 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:11.148515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1789dd5-1d9b-475f-8b10-91b0214e3586 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:11.240561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a701cc-5075-4860-870b-fe57f0321914 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:11.265361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:11.266266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 91/182] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:11.267138 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1789dd5-1d9b-475f-8b10-91b0214e3586 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:11.281994 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce955a-0a56-40a7-b186-acad5a3904a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:11.282816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce955a-0a56-40a7-b186-acad5a3904a9 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:11.283098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce955a-0a56-40a7-b186-acad5a3904a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:11.283377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce955a-0a56-40a7-b186-acad5a3904a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:11.295054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce955a-0a56-40a7-b186-acad5a3904a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:11.303217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-bcce955a-0a56-40a7-b186-acad5a3904a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:11.303217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:11.303217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:11.303217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:11.303217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:11.303217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:11.303217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:11.303217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:11.303217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:11.303217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:11.303217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:11.303217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:11.303217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:11.303217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:11.303217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:11.303217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:11.304086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:11.304086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:11.304086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:11.304086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:11.304086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:11.304086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:11.304086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:11.304086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:11.304086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:11.304086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:11.304086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:11.304086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:11.304086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:11.304086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:11.304086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:11.304086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:11.304910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:11.304910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:11.304910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:11.304910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:11.304910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:11.304910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:11.304910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:11.304910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:11.304910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:11.304910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:11.304910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:11.304910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:11.304910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 92/183] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:52:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:11.305548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce955a-0a56-40a7-b186-acad5a3904a9 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:11.377551 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:11.393146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1789dd5-1d9b-475f-8b10-91b0214e3586 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:11.416274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce955a-0a56-40a7-b186-acad5a3904a9 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:11.428602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded5d80-a55d-4fae-99a4-e4fcb0a6d926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:11.429287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded5d80-a55d-4fae-99a4-e4fcb0a6d926 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:11.429580 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded5d80-a55d-4fae-99a4-e4fcb0a6d926 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:11.429857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded5d80-a55d-4fae-99a4-e4fcb0a6d926 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:11.455610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b7abd-27a5-4c71-8152-c8295987d094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:11.456320 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b7abd-27a5-4c71-8152-c8295987d094 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:11.456561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b7abd-27a5-4c71-8152-c8295987d094 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:11.456868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b7abd-27a5-4c71-8152-c8295987d094 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:11.457627 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b7abd-27a5-4c71-8152-c8295987d094 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:11.457933 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b7abd-27a5-4c71-8152-c8295987d094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:11.459461 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b7abd-27a5-4c71-8152-c8295987d094 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:11.469645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 93/184] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:11.473719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b7abd-27a5-4c71-8152-c8295987d094 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:11.503255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded5d80-a55d-4fae-99a4-e4fcb0a6d926 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:11.504741 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded5d80-a55d-4fae-99a4-e4fcb0a6d926 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:11.523876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:11.593357 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b7abd-27a5-4c71-8152-c8295987d094 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:11.598543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:11.614200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 92/185] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:11.615030 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded5d80-a55d-4fae-99a4-e4fcb0a6d926 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:11.634767 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b3e3f-7102-44ab-a176-708b70a0f0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:11.635490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b3e3f-7102-44ab-a176-708b70a0f0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:11.635773 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b3e3f-7102-44ab-a176-708b70a0f0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:11.636048 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b3e3f-7102-44ab-a176-708b70a0f0f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:11.637339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b3e3f-7102-44ab-a176-708b70a0f0f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:11.638799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b3e3f-7102-44ab-a176-708b70a0f0f2 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:11.640902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 94/186] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:11.641453 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b3e3f-7102-44ab-a176-708b70a0f0f2 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:11.734303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded5d80-a55d-4fae-99a4-e4fcb0a6d926 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:11.749864 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b3e3f-7102-44ab-a176-708b70a0f0f2 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:11.765015 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:11.772346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847279c1-eb87-4354-8ff4-8ed9727aa4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:11.773674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847279c1-eb87-4354-8ff4-8ed9727aa4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:11.774122 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847279c1-eb87-4354-8ff4-8ed9727aa4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:11.774415 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847279c1-eb87-4354-8ff4-8ed9727aa4f6 None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:11.776332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847279c1-eb87-4354-8ff4-8ed9727aa4f6 None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:11.777852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847279c1-eb87-4354-8ff4-8ed9727aa4f6 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:11.784141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 93/187] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:11 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:11.784141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847279c1-eb87-4354-8ff4-8ed9727aa4f6 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:11.793121 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809d8bd-45a4-4597-9974-734835d46702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:11.793807 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809d8bd-45a4-4597-9974-734835d46702 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:11.794095 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809d8bd-45a4-4597-9974-734835d46702 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:11.794375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809d8bd-45a4-4597-9974-734835d46702 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:11.795128 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809d8bd-45a4-4597-9974-734835d46702 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:11.795434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809d8bd-45a4-4597-9974-734835d46702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:11.796880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809d8bd-45a4-4597-9974-734835d46702 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:11.805258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 95/188] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:52:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:11.805900 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809d8bd-45a4-4597-9974-734835d46702 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:11.852951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:11.905276 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847279c1-eb87-4354-8ff4-8ed9727aa4f6 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:11.919494 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809d8bd-45a4-4597-9974-734835d46702 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:11.943171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb58f33-5c1f-4368-a45e-075772c9bed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:11.944054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb58f33-5c1f-4368-a45e-075772c9bed9 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:11.944542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb58f33-5c1f-4368-a45e-075772c9bed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:11.944663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb58f33-5c1f-4368-a45e-075772c9bed9 None admin] PATH_INFO: `/v3/users/00c92ff7818146a5929c9078f3dcf87f` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:11.956347 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9589bab-11fe-403a-aebd-4cee6a337a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:11.960409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9589bab-11fe-403a-aebd-4cee6a337a17 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:11.960409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9589bab-11fe-403a-aebd-4cee6a337a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:11.960409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9589bab-11fe-403a-aebd-4cee6a337a17 None admin] PATH_INFO: `/v3/projects/1693b44e5fea4067af432358e61ee9f5` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:11.963517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9589bab-11fe-403a-aebd-4cee6a337a17 None admin] RBAC: Authorizing `identity:get_project(project_id=1693b44e5fea4067af432358e61ee9f5)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:11.965498 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9589bab-11fe-403a-aebd-4cee6a337a17 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:11.971922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 96/189] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:11 2022] GET /identity/v3/projects/1693b44e5fea4067af432358e61ee9f5 => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:11.971922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9589bab-11fe-403a-aebd-4cee6a337a17 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:11.972640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb58f33-5c1f-4368-a45e-075772c9bed9 None admin] RBAC: Authorizing `identity:get_user(user_id=00c92ff7818146a5929c9078f3dcf87f)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:11.974524 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb58f33-5c1f-4368-a45e-075772c9bed9 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:11.977271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 94/190] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:11 2022] GET /identity/v3/users/00c92ff7818146a5929c9078f3dcf87f => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:11.978586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb58f33-5c1f-4368-a45e-075772c9bed9 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:12.019330 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:12.086009 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9589bab-11fe-403a-aebd-4cee6a337a17 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:12.090551 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb58f33-5c1f-4368-a45e-075772c9bed9 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:12.105851 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:12.123592 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81141310-c4f6-4656-a5e3-b13879be109d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:12.124386 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81141310-c4f6-4656-a5e3-b13879be109d None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:12.124650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81141310-c4f6-4656-a5e3-b13879be109d None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:12.124956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81141310-c4f6-4656-a5e3-b13879be109d None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:12.125661 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81141310-c4f6-4656-a5e3-b13879be109d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:12.127117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81141310-c4f6-4656-a5e3-b13879be109d None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:12.127980 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987431ce-6a77-467b-bdc2-85ebc08e1386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:12.128772 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987431ce-6a77-467b-bdc2-85ebc08e1386 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:12.129058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987431ce-6a77-467b-bdc2-85ebc08e1386 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:12.129337 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987431ce-6a77-467b-bdc2-85ebc08e1386 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:12.130293 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987431ce-6a77-467b-bdc2-85ebc08e1386 None admin] RBAC: Adding query filter params (role.id=b5ee21738ecb4079977d078d50ddbcf9, scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5, user.id=f0a92aa0e8164ee0890c4bb9a039faf7) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:12.130610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987431ce-6a77-467b-bdc2-85ebc08e1386 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:12.132075 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987431ce-6a77-467b-bdc2-85ebc08e1386 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:12.139262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 95/191] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:11 2022] GET /identity/v3/role_assignments?user.id=f0a92aa0e8164ee0890c4bb9a039faf7&scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5&role.id=b5ee21738ecb4079977d078d50ddbcf9 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:12.139972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987431ce-6a77-467b-bdc2-85ebc08e1386 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:12.167921 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 97/192] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:52:11 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:52:12.169731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81141310-c4f6-4656-a5e3-b13879be109d None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:12.272949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987431ce-6a77-467b-bdc2-85ebc08e1386 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:12.323626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbc7e3-710e-4bdb-9ee1-76eda048a64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:12.325704 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbc7e3-710e-4bdb-9ee1-76eda048a64c None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:12.326437 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbc7e3-710e-4bdb-9ee1-76eda048a64c None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:12.328226 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:12.331092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbc7e3-710e-4bdb-9ee1-76eda048a64c None admin] PATH_INFO: `/v3/users/00c92ff7818146a5929c9078f3dcf87f` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:12.333888 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbc7e3-710e-4bdb-9ee1-76eda048a64c None admin] RBAC: Authorizing `identity:get_user(user_id=00c92ff7818146a5929c9078f3dcf87f)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:12.336141 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81141310-c4f6-4656-a5e3-b13879be109d None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:12.338813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbc7e3-710e-4bdb-9ee1-76eda048a64c None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:12.349311 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 96/193] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:12 2022] GET /identity/v3/users/00c92ff7818146a5929c9078f3dcf87f => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:12.350547 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbc7e3-710e-4bdb-9ee1-76eda048a64c None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:12.411894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708726b8-5365-4af3-a372-04ee0cf0af5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:12.413164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708726b8-5365-4af3-a372-04ee0cf0af5b None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:12.413770 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708726b8-5365-4af3-a372-04ee0cf0af5b None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:12.414252 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708726b8-5365-4af3-a372-04ee0cf0af5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:12.415338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708726b8-5365-4af3-a372-04ee0cf0af5b None admin] RBAC: Adding query filter params (role.id=b5ee21738ecb4079977d078d50ddbcf9, scope.project.id=1693b44e5fea4067af432358e61ee9f5, user.id=f0a92aa0e8164ee0890c4bb9a039faf7) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:12.415980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708726b8-5365-4af3-a372-04ee0cf0af5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:12.418084 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:12.418774 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708726b8-5365-4af3-a372-04ee0cf0af5b None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:12.430787 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 98/194] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:12 2022] GET /identity/v3/role_assignments?user.id=f0a92aa0e8164ee0890c4bb9a039faf7&scope.project.id=1693b44e5fea4067af432358e61ee9f5&role.id=b5ee21738ecb4079977d078d50ddbcf9 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:12.433293 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708726b8-5365-4af3-a372-04ee0cf0af5b None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:12.521985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbc7e3-710e-4bdb-9ee1-76eda048a64c None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:12.572669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc74f3-5769-40f4-9b7c-1fafd2a0d546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:12.573828 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc74f3-5769-40f4-9b7c-1fafd2a0d546 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:12.574311 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc74f3-5769-40f4-9b7c-1fafd2a0d546 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:12.574779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc74f3-5769-40f4-9b7c-1fafd2a0d546 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:12.595069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708726b8-5365-4af3-a372-04ee0cf0af5b None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:12.646627 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247889e-685f-4206-985c-91a273cde46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:12.647910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247889e-685f-4206-985c-91a273cde46b None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:12.648203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247889e-685f-4206-985c-91a273cde46b None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:12.648485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247889e-685f-4206-985c-91a273cde46b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:12.670502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc74f3-5769-40f4-9b7c-1fafd2a0d546 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:12.672395 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc74f3-5769-40f4-9b7c-1fafd2a0d546 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:12.690369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:12.736532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:12.746245 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247889e-685f-4206-985c-91a273cde46b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:12.748042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247889e-685f-4206-985c-91a273cde46b None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:12.828383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 97/195] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:12.835102 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc74f3-5769-40f4-9b7c-1fafd2a0d546 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:12.892434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 99/196] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:12.894013 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b955b6b-7691-4f5d-abaf-ef30191c641d None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:12.894307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b955b6b-7691-4f5d-abaf-ef30191c641d None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:12.894585 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b955b6b-7691-4f5d-abaf-ef30191c641d None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:12.896013 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 100/197] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:12.900645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b955b6b-7691-4f5d-abaf-ef30191c641d None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:12.976380 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc74f3-5769-40f4-9b7c-1fafd2a0d546 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:12.979661 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:13.013149 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcc274-7024-4843-b064-1df83d289d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:13.014219 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcc274-7024-4843-b064-1df83d289d6a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:13.014567 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcc274-7024-4843-b064-1df83d289d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:13.015125 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcc274-7024-4843-b064-1df83d289d6a None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:13.016374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcc274-7024-4843-b064-1df83d289d6a None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:13.017989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcc274-7024-4843-b064-1df83d289d6a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:13.019984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 98/198] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:12 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:13.024732 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86fe93-6336-420a-9070-b3d7be2dc9f7 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:13.024732 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86fe93-6336-420a-9070-b3d7be2dc9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:13.024732 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86fe93-6336-420a-9070-b3d7be2dc9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:13.031450 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b955b6b-7691-4f5d-abaf-ef30191c641d None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:13.040074 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-ab86fe93-6336-420a-9070-b3d7be2dc9f7 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:13.041334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ab86fe93-6336-420a-9070-b3d7be2dc9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:13.051843 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:13.070124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf687ca7-ccb5-4a03-8403-72db0da5b387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:13.071272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf687ca7-ccb5-4a03-8403-72db0da5b387 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:13.072242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf687ca7-ccb5-4a03-8403-72db0da5b387 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:13.072838 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf687ca7-ccb5-4a03-8403-72db0da5b387 None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:13.074212 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf687ca7-ccb5-4a03-8403-72db0da5b387 None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:13.075837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf687ca7-ccb5-4a03-8403-72db0da5b387 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:13.077792 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 101/199] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:12 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:13.078703 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf687ca7-ccb5-4a03-8403-72db0da5b387 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:13.115860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 99/200] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:13.121432 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86fe93-6336-420a-9070-b3d7be2dc9f7 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:13.190954 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf687ca7-ccb5-4a03-8403-72db0da5b387 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:13.234043 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:13.244454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9953d-71b0-437a-8453-266b15aa6e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:13.245724 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9953d-71b0-437a-8453-266b15aa6e40 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:13.246159 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9953d-71b0-437a-8453-266b15aa6e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:13.246583 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9953d-71b0-437a-8453-266b15aa6e40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:13.247591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f9953d-71b0-437a-8453-266b15aa6e40 None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.project.id=bbe948d5cac04aeb909127cea42f77b4, user.id=00c92ff7818146a5929c9078f3dcf87f) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:13.248653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f9953d-71b0-437a-8453-266b15aa6e40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:13.250550 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f9953d-71b0-437a-8453-266b15aa6e40 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:13.264893 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 102/201] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:13 2022] GET /identity/v3/role_assignments?user.id=00c92ff7818146a5929c9078f3dcf87f&scope.project.id=bbe948d5cac04aeb909127cea42f77b4&role.id=053d1730ed764973abb1490894e07681 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:13.264893 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9953d-71b0-437a-8453-266b15aa6e40 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:13.266984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86fe93-6336-420a-9070-b3d7be2dc9f7 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:13.326803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581928a6-57d2-42bf-993a-5bb15c711983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:13.328800 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581928a6-57d2-42bf-993a-5bb15c711983 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:13.329159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581928a6-57d2-42bf-993a-5bb15c711983 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:13.329442 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581928a6-57d2-42bf-993a-5bb15c711983 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:13.330330 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581928a6-57d2-42bf-993a-5bb15c711983 None admin] RBAC: Adding query filter params (role.id=160d4162340e472b9712a7f15cb2ecca, scope.project.id=bbe948d5cac04aeb909127cea42f77b4, user.id=00c92ff7818146a5929c9078f3dcf87f) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:13.330648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581928a6-57d2-42bf-993a-5bb15c711983 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:13.335910 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581928a6-57d2-42bf-993a-5bb15c711983 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:13.349998 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 100/202] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:13 2022] GET /identity/v3/role_assignments?user.id=00c92ff7818146a5929c9078f3dcf87f&scope.project.id=bbe948d5cac04aeb909127cea42f77b4&role.id=160d4162340e472b9712a7f15cb2ecca => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:13.430721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9953d-71b0-437a-8453-266b15aa6e40 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:13.480537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa690ba7-b45e-4ad7-a4a6-3b44ab83c88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:13.481655 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa690ba7-b45e-4ad7-a4a6-3b44ab83c88f None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:13.482096 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa690ba7-b45e-4ad7-a4a6-3b44ab83c88f None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:13.482538 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa690ba7-b45e-4ad7-a4a6-3b44ab83c88f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:13.496542 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa690ba7-b45e-4ad7-a4a6-3b44ab83c88f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:13.507633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-fa690ba7-b45e-4ad7-a4a6-3b44ab83c88f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:13.507633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:13.507633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:13.507633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:13.507633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:13.507633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:13.507633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:13.507633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:13.507633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:13.507633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:13.507633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:13.507633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:13.507633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:13.507633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:13.507633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:13.507633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:13.508789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:13.508789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:13.508789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:13.508789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:13.508789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:13.508789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:13.508789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:13.508789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:13.508789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:13.508789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:13.508789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:13.508789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:13.508789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:13.508789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:13.508789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:13.508789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:13.510610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:13.510610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:13.510610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:13.510610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:13.510610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:13.510610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:13.510610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:13.510610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:13.510610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:13.510610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:13.510610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:13.510610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:13.512808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 103/203] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:13.521308 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581928a6-57d2-42bf-993a-5bb15c711983 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:13.539066 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:13.666373 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581928a6-57d2-42bf-993a-5bb15c711983 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:13.714325 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00786259-e7e8-4ffd-ac29-fc861bb3a259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:13.715004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00786259-e7e8-4ffd-ac29-fc861bb3a259 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:13.715289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00786259-e7e8-4ffd-ac29-fc861bb3a259 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:13.715567 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00786259-e7e8-4ffd-ac29-fc861bb3a259 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:13.716324 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00786259-e7e8-4ffd-ac29-fc861bb3a259 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:13.716632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00786259-e7e8-4ffd-ac29-fc861bb3a259 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:13.718060 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00786259-e7e8-4ffd-ac29-fc861bb3a259 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:13.733574 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 101/204] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:13.739568 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa690ba7-b45e-4ad7-a4a6-3b44ab83c88f None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:13.828626 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:13.902146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa690ba7-b45e-4ad7-a4a6-3b44ab83c88f None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:13.963603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe92eda-65f6-4ea7-891b-3fb4c4c1b480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:13.965323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe92eda-65f6-4ea7-891b-3fb4c4c1b480 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:13.965323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe92eda-65f6-4ea7-891b-3fb4c4c1b480 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:13.965485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe92eda-65f6-4ea7-891b-3fb4c4c1b480 None admin] PATH_INFO: `/v3/users/54524a6e8932472a80eae538a8b3a26c` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:13.972586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe92eda-65f6-4ea7-891b-3fb4c4c1b480 None admin] RBAC: Authorizing `identity:get_user(user_id=54524a6e8932472a80eae538a8b3a26c)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:13.972586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe92eda-65f6-4ea7-891b-3fb4c4c1b480 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:13.972795 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 104/205] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:13 2022] GET /identity/v3/users/54524a6e8932472a80eae538a8b3a26c => generated 289 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:13.979050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00786259-e7e8-4ffd-ac29-fc861bb3a259 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:14.081164 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:14.126943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00786259-e7e8-4ffd-ac29-fc861bb3a259 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:14.175326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f193e31-30a4-495e-b1ed-ad8a913b7a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:14.175991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f193e31-30a4-495e-b1ed-ad8a913b7a3f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:14.176275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f193e31-30a4-495e-b1ed-ad8a913b7a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:14.176558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f193e31-30a4-495e-b1ed-ad8a913b7a3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:14.276319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f193e31-30a4-495e-b1ed-ad8a913b7a3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:14.277876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f193e31-30a4-495e-b1ed-ad8a913b7a3f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:14.368060 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:14.410272 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 102/206] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:14.416429 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe92eda-65f6-4ea7-891b-3fb4c4c1b480 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:14.575158 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe92eda-65f6-4ea7-891b-3fb4c4c1b480 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:14.598747 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:14.623016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997c2ac-3c1a-4d4f-8972-8e0d11805c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:14.623922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997c2ac-3c1a-4d4f-8972-8e0d11805c17 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:14.624209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997c2ac-3c1a-4d4f-8972-8e0d11805c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:14.624555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997c2ac-3c1a-4d4f-8972-8e0d11805c17 None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:14.625815 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997c2ac-3c1a-4d4f-8972-8e0d11805c17 None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:14.627432 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997c2ac-3c1a-4d4f-8972-8e0d11805c17 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:14.633579 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 105/207] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:14 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:14.639556 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f193e31-30a4-495e-b1ed-ad8a913b7a3f None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:14.796939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f193e31-30a4-495e-b1ed-ad8a913b7a3f None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:14.849681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce20f16-d45c-4925-bb43-f8c2ea98f81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:14.850349 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce20f16-d45c-4925-bb43-f8c2ea98f81c None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:14.850633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce20f16-d45c-4925-bb43-f8c2ea98f81c None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:14.850914 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce20f16-d45c-4925-bb43-f8c2ea98f81c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:14.851763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce20f16-d45c-4925-bb43-f8c2ea98f81c None admin] RBAC: Adding query filter params (role.id=d43f8f61eb5f4489acfd5fba99b85919, scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5, user.id=54524a6e8932472a80eae538a8b3a26c) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:14.852077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce20f16-d45c-4925-bb43-f8c2ea98f81c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:14.855050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce20f16-d45c-4925-bb43-f8c2ea98f81c None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:14.865711 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 103/208] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:14 2022] GET /identity/v3/role_assignments?user.id=54524a6e8932472a80eae538a8b3a26c&scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5&role.id=d43f8f61eb5f4489acfd5fba99b85919 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:15.179585 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5f776-a9b1-4946-832e-1ab9bf836fa7 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:15.179948 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5f776-a9b1-4946-832e-1ab9bf836fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:15.180136 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5f776-a9b1-4946-832e-1ab9bf836fa7 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:15.181594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 106/209] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:15.186952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6e6dd-435d-4b4c-b5d0-0df9ad7708ee None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:15.187245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6e6dd-435d-4b4c-b5d0-0df9ad7708ee None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:15.187523 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6e6dd-435d-4b4c-b5d0-0df9ad7708ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:15.207603 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-4ed6e6dd-435d-4b4c-b5d0-0df9ad7708ee None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:15.208835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-4ed6e6dd-435d-4b4c-b5d0-0df9ad7708ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:15.239283 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:15.289101 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 104/210] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:15.310548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5f776-a9b1-4946-832e-1ab9bf836fa7 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:15.396396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186c828-9766-4448-a3c9-efcbe450d8a8 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:15.396924 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186c828-9766-4448-a3c9-efcbe450d8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:15.396996 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186c828-9766-4448-a3c9-efcbe450d8a8 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:15.398330 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 105/211] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:52:15.404285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661eb60-2875-47aa-81a1-0c81480e5407 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:15.404567 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661eb60-2875-47aa-81a1-0c81480e5407 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:15.404871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661eb60-2875-47aa-81a1-0c81480e5407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:15.433033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-7661eb60-2875-47aa-81a1-0c81480e5407 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:15.434404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-7661eb60-2875-47aa-81a1-0c81480e5407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:15.447905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5f776-a9b1-4946-832e-1ab9bf836fa7 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:15.499622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8540b17-dd46-4702-abca-435ff6a03c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:15.500837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8540b17-dd46-4702-abca-435ff6a03c5c None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:15.501326 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8540b17-dd46-4702-abca-435ff6a03c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:15.501796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8540b17-dd46-4702-abca-435ff6a03c5c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:15.517823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8540b17-dd46-4702-abca-435ff6a03c5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:15.521179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 106/212] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:52:15.527544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-a8540b17-dd46-4702-abca-435ff6a03c5c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:15.527544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:15.527544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:15.527544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:15.527544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:15.527544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:15.527544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:15.527544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:15.527544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:15.527544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:15.527544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:15.527544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:15.527544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:15.527544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:15.527544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:15.527544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:15.528426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:15.528426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:15.528426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:15.528426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:15.528426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:15.528426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:15.528426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:15.528426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:15.528426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:15.528426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:15.528426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:15.528426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:15.528426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:15.528426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:15.528426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:15.528426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:15.529250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:15.529250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:15.529250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:15.529250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:15.529250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:15.529250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:15.529250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:15.529250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:15.529250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:15.529250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:15.529250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:15.529250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:15.531931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 107/213] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:15.540105 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661eb60-2875-47aa-81a1-0c81480e5407 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:15.547588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8540b17-dd46-4702-abca-435ff6a03c5c None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:15.614542 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:15.647376 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:15.657839 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661eb60-2875-47aa-81a1-0c81480e5407 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:15.686122 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8540b17-dd46-4702-abca-435ff6a03c5c None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:15.695850 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57efe6-159b-4a4d-8d46-ef273641d3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:15.697016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57efe6-159b-4a4d-8d46-ef273641d3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:15.697776 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57efe6-159b-4a4d-8d46-ef273641d3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:15.698315 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57efe6-159b-4a4d-8d46-ef273641d3e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:15.699217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57efe6-159b-4a4d-8d46-ef273641d3e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:15.699700 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57efe6-159b-4a4d-8d46-ef273641d3e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:15.701127 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57efe6-159b-4a4d-8d46-ef273641d3e5 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:15.711213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 107/214] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:15.717833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57efe6-159b-4a4d-8d46-ef273641d3e5 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:15.731378 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddf1c0-e801-45a5-b980-e73caf48f807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:15.732522 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddf1c0-e801-45a5-b980-e73caf48f807 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:15.732999 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddf1c0-e801-45a5-b980-e73caf48f807 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:15.733286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddf1c0-e801-45a5-b980-e73caf48f807 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:15.734493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddf1c0-e801-45a5-b980-e73caf48f807 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:15.735967 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddf1c0-e801-45a5-b980-e73caf48f807 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:15.737773 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 108/215] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:15.738984 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e86c51-17df-484a-8ca2-5464000f1da4 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:15.739253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e86c51-17df-484a-8ca2-5464000f1da4 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:15.739534 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e86c51-17df-484a-8ca2-5464000f1da4 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:15.740943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 109/216] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:15.745574 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e86c51-17df-484a-8ca2-5464000f1da4 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:15.836157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57efe6-159b-4a4d-8d46-ef273641d3e5 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:15.861860 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e86c51-17df-484a-8ca2-5464000f1da4 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:15.872389 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc120be-c3eb-40d8-9ec3-28e766ae6dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:15.873310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc120be-c3eb-40d8-9ec3-28e766ae6dbc None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:15.873594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc120be-c3eb-40d8-9ec3-28e766ae6dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:15.873868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc120be-c3eb-40d8-9ec3-28e766ae6dbc None admin] PATH_INFO: `/v3/users/c8e616111b884178bca12f6877a18f82` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:15.887538 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:15.888463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc120be-c3eb-40d8-9ec3-28e766ae6dbc None admin] RBAC: Authorizing `identity:get_user(user_id=c8e616111b884178bca12f6877a18f82)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:15.890090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc120be-c3eb-40d8-9ec3-28e766ae6dbc None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:15.891841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 108/217] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:15 2022] GET /identity/v3/users/c8e616111b884178bca12f6877a18f82 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:15.893116 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f1fb2-db2b-40fc-ac1d-170ebce2bf65 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:15.893401 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f1fb2-db2b-40fc-ac1d-170ebce2bf65 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:15.893677 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f1fb2-db2b-40fc-ac1d-170ebce2bf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:15.910789 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-0b2f1fb2-db2b-40fc-ac1d-170ebce2bf65 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:15.911108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f34e6-39fa-4c18-9dfb-8ecb484540a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:15.911776 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f34e6-39fa-4c18-9dfb-8ecb484540a3 None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:15.911981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0b2f1fb2-db2b-40fc-ac1d-170ebce2bf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:15.912082 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f34e6-39fa-4c18-9dfb-8ecb484540a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:15.912312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f34e6-39fa-4c18-9dfb-8ecb484540a3 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:15.913058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f34e6-39fa-4c18-9dfb-8ecb484540a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:15.914495 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f34e6-39fa-4c18-9dfb-8ecb484540a3 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:15.922771 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:15.935963 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 110/218] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:52:15 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:52:15.937306 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a17fb-b905-4305-948f-d9f52e13e69e None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:15.937599 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a17fb-b905-4305-948f-d9f52e13e69e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:15.938119 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a17fb-b905-4305-948f-d9f52e13e69e None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:15.939295 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 111/219] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:15.944450 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a17fb-b905-4305-948f-d9f52e13e69e None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:15.997149 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 109/220] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:15.998475 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80667cc1-4885-42c4-b7f0-c4f68d862097 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:15.998767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80667cc1-4885-42c4-b7f0-c4f68d862097 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:15.999041 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80667cc1-4885-42c4-b7f0-c4f68d862097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:16.018210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-80667cc1-4885-42c4-b7f0-c4f68d862097 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:16.019535 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-80667cc1-4885-42c4-b7f0-c4f68d862097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:16.082848 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a17fb-b905-4305-948f-d9f52e13e69e None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:16.092854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 110/221] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:16.093538 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80667cc1-4885-42c4-b7f0-c4f68d862097 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:16.127382 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebcb51-be99-49f2-9512-89f1bf13bb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:16.128365 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebcb51-be99-49f2-9512-89f1bf13bb6c None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:16.128651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebcb51-be99-49f2-9512-89f1bf13bb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:16.129005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebcb51-be99-49f2-9512-89f1bf13bb6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:16.178605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:16.179715 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:16.215766 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80667cc1-4885-42c4-b7f0-c4f68d862097 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:16.217491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebcb51-be99-49f2-9512-89f1bf13bb6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:16.219385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebcb51-be99-49f2-9512-89f1bf13bb6c None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:16.262889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e3184-061a-440b-9bb2-dc5a7a6f80f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:16.263699 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e3184-061a-440b-9bb2-dc5a7a6f80f5 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:16.263977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e3184-061a-440b-9bb2-dc5a7a6f80f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:16.264285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e3184-061a-440b-9bb2-dc5a7a6f80f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:16.277822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e3184-061a-440b-9bb2-dc5a7a6f80f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:16.285757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-e64e3184-061a-440b-9bb2-dc5a7a6f80f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:52:16.285757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:16.285757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:16.285757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:16.285757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:16.285757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:16.285757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:16.285757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:16.285757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:16.285757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:16.285757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:16.285757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:16.285757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:16.285757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:16.285757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:16.285757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:16.286668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:16.286668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:16.286668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:16.286668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:16.286668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:16.286668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:16.286668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:16.286668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:16.286668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:16.286668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:16.286668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:16.286668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:16.286668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:16.286668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:16.286668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:16.286668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:16.287455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:16.287455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:16.287455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:16.287455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:16.287455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:16.287455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:16.287455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:16.287455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:16.287455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:16.287455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:16.287455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:52:16.287455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:16.287455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 111/222] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:52:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:16.288110 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e3184-061a-440b-9bb2-dc5a7a6f80f5 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:16.353068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 112/223] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:16.354168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebcb51-be99-49f2-9512-89f1bf13bb6c None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:16.406208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e3184-061a-440b-9bb2-dc5a7a6f80f5 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:16.445115 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b0fee-82e5-4aa4-9d29-e35f4991885d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:16.446076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b0fee-82e5-4aa4-9d29-e35f4991885d None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:16.446478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b0fee-82e5-4aa4-9d29-e35f4991885d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:16.446961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b0fee-82e5-4aa4-9d29-e35f4991885d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:16.449363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:16.461482 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4b0fee-82e5-4aa4-9d29-e35f4991885d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:16.466208 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:16.470719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-6f4b0fee-82e5-4aa4-9d29-e35f4991885d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:16.470719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:16.470719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:16.470719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:16.470719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:16.470719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:16.470719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:16.470719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:16.470719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:16.470719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:16.470719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:16.470719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:16.470719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:16.470719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:16.470719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:16.470719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:16.472163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:16.472163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:16.472163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:16.472163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:16.472163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:16.472163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:16.472163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:16.472163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:16.472163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:16.472163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:16.472163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:16.472163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:16.472163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:16.472163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:16.472163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:16.472163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:16.473463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:16.473463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:16.473463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:16.473463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:16.473463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:16.473463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:16.473463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:16.473463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:16.473463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:16.473463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:16.473463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:16.473463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:16.474788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 112/224] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:52:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:16.475674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b0fee-82e5-4aa4-9d29-e35f4991885d None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:16.508990 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebcb51-be99-49f2-9512-89f1bf13bb6c None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:16.562487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c75c1-54d5-4089-9a30-046e70878610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:16.563438 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c75c1-54d5-4089-9a30-046e70878610 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:16.563872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c75c1-54d5-4089-9a30-046e70878610 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:16.564296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c75c1-54d5-4089-9a30-046e70878610 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:16.565407 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c75c1-54d5-4089-9a30-046e70878610 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:16.567782 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c75c1-54d5-4089-9a30-046e70878610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:16.569405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c75c1-54d5-4089-9a30-046e70878610 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:16.578902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 113/225] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:16.579971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c75c1-54d5-4089-9a30-046e70878610 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:16.593883 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b0fee-82e5-4aa4-9d29-e35f4991885d None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:16.633857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ebbf7-e02a-4de7-971d-2147cef73f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:16.635629 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ebbf7-e02a-4de7-971d-2147cef73f2c None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:16.636098 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ebbf7-e02a-4de7-971d-2147cef73f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:16.636522 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ebbf7-e02a-4de7-971d-2147cef73f2c None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:16.637933 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ebbf7-e02a-4de7-971d-2147cef73f2c None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:16.639680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ebbf7-e02a-4de7-971d-2147cef73f2c None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:16.641641 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 113/226] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:16 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:16.642687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ebbf7-e02a-4de7-971d-2147cef73f2c None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:16.706877 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c75c1-54d5-4089-9a30-046e70878610 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:16.742067 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:16.751658 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcc036-6784-4a4e-986b-0ee3927af7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:16.752342 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcc036-6784-4a4e-986b-0ee3927af7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:16.752791 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcc036-6784-4a4e-986b-0ee3927af7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:16.753216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcc036-6784-4a4e-986b-0ee3927af7c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:16.753985 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfcc036-6784-4a4e-986b-0ee3927af7c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:16.754296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfcc036-6784-4a4e-986b-0ee3927af7c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:16.756619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfcc036-6784-4a4e-986b-0ee3927af7c1 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:16.760152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:16.767683 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ebbf7-e02a-4de7-971d-2147cef73f2c None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:16.773107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 114/227] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:52:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:16.775116 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcc036-6784-4a4e-986b-0ee3927af7c1 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:16.816539 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93d8ff-aa03-4c1e-8daf-6e8ea1522966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:16.817579 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93d8ff-aa03-4c1e-8daf-6e8ea1522966 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:16.817923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93d8ff-aa03-4c1e-8daf-6e8ea1522966 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:16.818210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93d8ff-aa03-4c1e-8daf-6e8ea1522966 None admin] PATH_INFO: `/v3/users/00c92ff7818146a5929c9078f3dcf87f` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:16.819479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93d8ff-aa03-4c1e-8daf-6e8ea1522966 None admin] RBAC: Authorizing `identity:get_user(user_id=00c92ff7818146a5929c9078f3dcf87f)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:16.822498 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93d8ff-aa03-4c1e-8daf-6e8ea1522966 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:16.835259 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 114/228] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:16 2022] GET /identity/v3/users/00c92ff7818146a5929c9078f3dcf87f => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:16.835259 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93d8ff-aa03-4c1e-8daf-6e8ea1522966 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:16.898374 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcc036-6784-4a4e-986b-0ee3927af7c1 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:16.941752 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bd205-b63a-44b8-9a03-ffcbf515f6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:16.942768 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bd205-b63a-44b8-9a03-ffcbf515f6cb None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:16.943284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bd205-b63a-44b8-9a03-ffcbf515f6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:16.943766 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bd205-b63a-44b8-9a03-ffcbf515f6cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:16.944849 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93bd205-b63a-44b8-9a03-ffcbf515f6cb None admin] RBAC: Adding query filter params (role.id=b5ee21738ecb4079977d078d50ddbcf9, scope.project.id=bbe948d5cac04aeb909127cea42f77b4, user.id=c8e616111b884178bca12f6877a18f82) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:16.945398 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93bd205-b63a-44b8-9a03-ffcbf515f6cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:16.947823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93bd205-b63a-44b8-9a03-ffcbf515f6cb None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:16.954187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 115/229] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:16 2022] GET /identity/v3/role_assignments?user.id=c8e616111b884178bca12f6877a18f82&scope.project.id=bbe948d5cac04aeb909127cea42f77b4&role.id=b5ee21738ecb4079977d078d50ddbcf9 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:16.955285 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bd205-b63a-44b8-9a03-ffcbf515f6cb None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:16.972237 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93d8ff-aa03-4c1e-8daf-6e8ea1522966 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:17.011001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:17.012467 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b85d7b-ab8d-4663-88dd-c8ada4ed1e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:17.013612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b85d7b-ab8d-4663-88dd-c8ada4ed1e5d None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:17.014034 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b85d7b-ab8d-4663-88dd-c8ada4ed1e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:17.014419 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b85d7b-ab8d-4663-88dd-c8ada4ed1e5d None admin] PATH_INFO: `/v3/users/00c92ff7818146a5929c9078f3dcf87f` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:17.016361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b85d7b-ab8d-4663-88dd-c8ada4ed1e5d None admin] RBAC: Authorizing `identity:get_user(user_id=00c92ff7818146a5929c9078f3dcf87f)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:17.018077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b85d7b-ab8d-4663-88dd-c8ada4ed1e5d None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:17.020280 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 115/230] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:16 2022] GET /identity/v3/users/00c92ff7818146a5929c9078f3dcf87f => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:17.026229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b85d7b-ab8d-4663-88dd-c8ada4ed1e5d None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:17.026893 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:17.073280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bd205-b63a-44b8-9a03-ffcbf515f6cb None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:17.109908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67d1a7-62e6-4695-b8c9-ee90a4c101f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:17.111076 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67d1a7-62e6-4695-b8c9-ee90a4c101f8 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:17.111511 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67d1a7-62e6-4695-b8c9-ee90a4c101f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:17.111934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67d1a7-62e6-4695-b8c9-ee90a4c101f8 None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:17.113440 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67d1a7-62e6-4695-b8c9-ee90a4c101f8 None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:17.115108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67d1a7-62e6-4695-b8c9-ee90a4c101f8 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:17.117223 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 116/231] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:16 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:17.131389 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67d1a7-62e6-4695-b8c9-ee90a4c101f8 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:17.169895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b85d7b-ab8d-4663-88dd-c8ada4ed1e5d None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:17.232903 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d7633-0277-4c98-99df-e5e2f38df232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:17.234660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d7633-0277-4c98-99df-e5e2f38df232 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:17.234660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d7633-0277-4c98-99df-e5e2f38df232 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:17.234660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d7633-0277-4c98-99df-e5e2f38df232 None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:17.236005 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d7633-0277-4c98-99df-e5e2f38df232 None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:17.238936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d7633-0277-4c98-99df-e5e2f38df232 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:17.242168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 116/232] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:17 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:17.251134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d7633-0277-4c98-99df-e5e2f38df232 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:17.307500 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67d1a7-62e6-4695-b8c9-ee90a4c101f8 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:17.343213 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:17.354174 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3be378-1573-4b4e-b519-e8ce2412cb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:17.356307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3be378-1573-4b4e-b519-e8ce2412cb89 None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:17.357017 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3be378-1573-4b4e-b519-e8ce2412cb89 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:17.357588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3be378-1573-4b4e-b519-e8ce2412cb89 None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4/users/00c92ff7818146a5929c9078f3dcf87f/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:17.359982 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3be378-1573-4b4e-b519-e8ce2412cb89 None admin] RBAC: Authorizing `identity:create_grant(project_id=bbe948d5cac04aeb909127cea42f77b4, user_id=00c92ff7818146a5929c9078f3dcf87f, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:17.362422 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3be378-1573-4b4e-b519-e8ce2412cb89 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:17.371837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:17.382250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 117/233] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:52:17 2022] PUT /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4/users/00c92ff7818146a5929c9078f3dcf87f/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:52:17.444306 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d7633-0277-4c98-99df-e5e2f38df232 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:17.497650 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed089220-b9c5-4f80-95be-d1b77c3b0a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:17.498591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed089220-b9c5-4f80-95be-d1b77c3b0a1f None admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:17.498877 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed089220-b9c5-4f80-95be-d1b77c3b0a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:17.499154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed089220-b9c5-4f80-95be-d1b77c3b0a1f None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4/users/00c92ff7818146a5929c9078f3dcf87f/roles/160d4162340e472b9712a7f15cb2ecca` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:17.501736 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed089220-b9c5-4f80-95be-d1b77c3b0a1f None admin] RBAC: Authorizing `identity:create_grant(project_id=bbe948d5cac04aeb909127cea42f77b4, user_id=00c92ff7818146a5929c9078f3dcf87f, role_id=160d4162340e472b9712a7f15cb2ecca)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:17.503239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed089220-b9c5-4f80-95be-d1b77c3b0a1f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:17.527655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 117/234] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:52:17 2022] PUT /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4/users/00c92ff7818146a5929c9078f3dcf87f/roles/160d4162340e472b9712a7f15cb2ecca => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 21:52:18.360322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd3b17-02be-4f1b-8318-e8b26f30f86e None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:18.360888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd3b17-02be-4f1b-8318-e8b26f30f86e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:18.361217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd3b17-02be-4f1b-8318-e8b26f30f86e None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:18.362718 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 118/235] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:18.368453 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbc256-bb98-4a71-9e97-a6f7c48245e4 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:18.369012 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbc256-bb98-4a71-9e97-a6f7c48245e4 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:18.369417 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbc256-bb98-4a71-9e97-a6f7c48245e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:18.387975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-81fbc256-bb98-4a71-9e97-a6f7c48245e4 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:18.389404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-81fbc256-bb98-4a71-9e97-a6f7c48245e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:18.491064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 118/236] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:18.527715 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd3b17-02be-4f1b-8318-e8b26f30f86e None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:18.660428 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd3b17-02be-4f1b-8318-e8b26f30f86e None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:18.711260 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01d49b-62d2-4c5e-8d3e-60b89cebec47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:18.712021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01d49b-62d2-4c5e-8d3e-60b89cebec47 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:18.712983 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01d49b-62d2-4c5e-8d3e-60b89cebec47 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:18.713271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01d49b-62d2-4c5e-8d3e-60b89cebec47 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:18.733604 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f01d49b-62d2-4c5e-8d3e-60b89cebec47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:18.741437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-5f01d49b-62d2-4c5e-8d3e-60b89cebec47 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:18.741437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:18.741437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:18.741437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:18.741437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:18.741437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:18.741437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:18.741437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:18.741437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:18.741437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:18.741437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:18.741437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:18.741437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:18.741437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:18.741437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:18.741437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:18.742334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:18.742334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:18.742334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:18.742334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:18.742334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:18.742334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:18.742334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:18.742334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:18.742334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:18.742334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:18.742334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:18.742334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:18.742334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:18.742334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:18.742334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:18.742334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:18.743111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:18.743111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:18.743111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:18.743111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:18.743111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:18.743111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:18.743111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:18.743111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:18.743111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:18.743111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:18.743111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:18.743111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:18.743111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 119/237] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:18.746193 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51318810-8acd-4b4c-bc54-8a2bac235ba9 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:18.746486 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51318810-8acd-4b4c-bc54-8a2bac235ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:18.746768 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51318810-8acd-4b4c-bc54-8a2bac235ba9 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:18.748212 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01d49b-62d2-4c5e-8d3e-60b89cebec47 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:18.748645 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 119/238] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:52:18.753543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdf000-7bfd-4f06-b60d-3ca6d4eb9916 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:18.753953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdf000-7bfd-4f06-b60d-3ca6d4eb9916 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:18.754340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdf000-7bfd-4f06-b60d-3ca6d4eb9916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:18.780633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-3ebdf000-7bfd-4f06-b60d-3ca6d4eb9916 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:18.781867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-3ebdf000-7bfd-4f06-b60d-3ca6d4eb9916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:18.857571 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:18.868157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:18.905845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 120/239] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:18.930304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebdf000-7bfd-4f06-b60d-3ca6d4eb9916 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:18.930733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01d49b-62d2-4c5e-8d3e-60b89cebec47 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:18.973208 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f40e4-00aa-412d-ac26-b8e4f994ab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:18.976931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f40e4-00aa-412d-ac26-b8e4f994ab31 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:18.977220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f40e4-00aa-412d-ac26-b8e4f994ab31 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:18.977501 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f40e4-00aa-412d-ac26-b8e4f994ab31 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:18.978856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f40e4-00aa-412d-ac26-b8e4f994ab31 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:18.979174 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f40e4-00aa-412d-ac26-b8e4f994ab31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:18.980627 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f40e4-00aa-412d-ac26-b8e4f994ab31 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:18.997738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 120/240] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:19.003353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f40e4-00aa-412d-ac26-b8e4f994ab31 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:19.116673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f40e4-00aa-412d-ac26-b8e4f994ab31 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:19.130479 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:19.159726 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09edad-c9af-41b6-8c0b-9ff5736e16ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:19.160876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09edad-c9af-41b6-8c0b-9ff5736e16ff None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:19.161311 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09edad-c9af-41b6-8c0b-9ff5736e16ff None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:19.161758 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09edad-c9af-41b6-8c0b-9ff5736e16ff None admin] PATH_INFO: `/v3/users/b979812232ec49879fa487443941740e` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:19.185612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebdf000-7bfd-4f06-b60d-3ca6d4eb9916 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:19.190166 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09edad-c9af-41b6-8c0b-9ff5736e16ff None admin] RBAC: Authorizing `identity:get_user(user_id=b979812232ec49879fa487443941740e)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:19.192071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09edad-c9af-41b6-8c0b-9ff5736e16ff None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:19.203876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 121/241] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:19 2022] GET /identity/v3/users/b979812232ec49879fa487443941740e => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:19.205625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09edad-c9af-41b6-8c0b-9ff5736e16ff None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:19.267077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e6d3c-95b5-4e66-ac9e-ab13389fc972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:19.268067 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e6d3c-95b5-4e66-ac9e-ab13389fc972 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:19.268186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e6d3c-95b5-4e66-ac9e-ab13389fc972 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:19.268403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e6d3c-95b5-4e66-ac9e-ab13389fc972 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:19.273523 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e6d3c-95b5-4e66-ac9e-ab13389fc972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:19.275407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e6d3c-95b5-4e66-ac9e-ab13389fc972 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:19.277427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 121/242] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:19.285591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e6d3c-95b5-4e66-ac9e-ab13389fc972 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:19.330781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09edad-c9af-41b6-8c0b-9ff5736e16ff None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:19.345122 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:19.376246 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05fdd8-7453-409a-a7e8-6b3d28833d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:19.377546 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05fdd8-7453-409a-a7e8-6b3d28833d29 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:19.377979 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05fdd8-7453-409a-a7e8-6b3d28833d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:19.378400 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05fdd8-7453-409a-a7e8-6b3d28833d29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:19.395085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e6d3c-95b5-4e66-ac9e-ab13389fc972 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:19.448588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:19.469331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c1a41-1b74-4f9e-a8d8-10b331fe5eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:19.470012 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c1a41-1b74-4f9e-a8d8-10b331fe5eb3 None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:19.470293 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c1a41-1b74-4f9e-a8d8-10b331fe5eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:19.470577 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c1a41-1b74-4f9e-a8d8-10b331fe5eb3 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:19.471397 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c1a41-1b74-4f9e-a8d8-10b331fe5eb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:19.475960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c1a41-1b74-4f9e-a8d8-10b331fe5eb3 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:19.493826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05fdd8-7453-409a-a7e8-6b3d28833d29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:19.495496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05fdd8-7453-409a-a7e8-6b3d28833d29 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:19.499745 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 122/243] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:52:19 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:52:19.501192 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee19e2c-0c1e-4c2b-aab8-67e32b1d3a45 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:19.501483 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee19e2c-0c1e-4c2b-aab8-67e32b1d3a45 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:19.501823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee19e2c-0c1e-4c2b-aab8-67e32b1d3a45 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:19.503201 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 123/244] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:19.509779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9aad8-08ee-4797-9650-7c103ee1bc10 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:19.510114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9aad8-08ee-4797-9650-7c103ee1bc10 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:19.510343 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9aad8-08ee-4797-9650-7c103ee1bc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:19.528082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-a8a9aad8-08ee-4797-9650-7c103ee1bc10 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:19.529457 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a8a9aad8-08ee-4797-9650-7c103ee1bc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:19.609188 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 124/245] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:52:19.621011 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 122/246] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:19.627485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9aad8-08ee-4797-9650-7c103ee1bc10 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:19.650334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05fdd8-7453-409a-a7e8-6b3d28833d29 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:19.680374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:19.748688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9aad8-08ee-4797-9650-7c103ee1bc10 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:19.782729 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:19.807403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ce284-c75b-40ef-9649-d2260ac80005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:19.808266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ce284-c75b-40ef-9649-d2260ac80005 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:19.808547 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ce284-c75b-40ef-9649-d2260ac80005 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:19.808851 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ce284-c75b-40ef-9649-d2260ac80005 None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:19.810657 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ce284-c75b-40ef-9649-d2260ac80005 None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:19.812220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ce284-c75b-40ef-9649-d2260ac80005 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:19.814299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 125/247] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:19 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:19.819380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05fdd8-7453-409a-a7e8-6b3d28833d29 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:19.820270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ce284-c75b-40ef-9649-d2260ac80005 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:19.866993 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c63a22-cd64-42ce-b46e-b3e876e537bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:19.868167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c63a22-cd64-42ce-b46e-b3e876e537bb None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:19.868555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c63a22-cd64-42ce-b46e-b3e876e537bb None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:19.868867 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c63a22-cd64-42ce-b46e-b3e876e537bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:19.885404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c63a22-cd64-42ce-b46e-b3e876e537bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:19.894004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-e7c63a22-cd64-42ce-b46e-b3e876e537bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:19.894004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:19.894004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:19.894004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:19.894004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:19.894004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:19.894004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:19.894004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:19.894004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:19.894004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:19.894004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:19.894004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:19.894004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:19.894004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:19.894004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:19.894004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:19.894952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:19.894952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:19.894952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:19.894952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:19.894952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:19.894952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:19.894952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:19.894952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:19.894952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:19.894952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:19.894952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:19.894952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:19.894952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:19.894952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:19.894952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:19.894952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:19.895735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:19.895735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:19.895735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:19.895735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:19.895735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:19.895735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:19.895735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:19.895735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:19.895735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:19.895735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:19.895735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:19.895735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:19.895735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 123/248] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:19.904591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c63a22-cd64-42ce-b46e-b3e876e537bb None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:19.999289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ce284-c75b-40ef-9649-d2260ac80005 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:20.013365 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:20.049877 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c63a22-cd64-42ce-b46e-b3e876e537bb None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:20.053218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afc12e-3812-4082-ae67-38ff8d0f77b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:20.056794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afc12e-3812-4082-ae67-38ff8d0f77b1 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:20.056794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afc12e-3812-4082-ae67-38ff8d0f77b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:20.056794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afc12e-3812-4082-ae67-38ff8d0f77b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:20.056794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afc12e-3812-4082-ae67-38ff8d0f77b1 None admin] RBAC: Adding query filter params (role.id=d43f8f61eb5f4489acfd5fba99b85919, scope.project.id=bbe948d5cac04aeb909127cea42f77b4, user.id=b979812232ec49879fa487443941740e) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:20.056794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afc12e-3812-4082-ae67-38ff8d0f77b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:20.058447 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afc12e-3812-4082-ae67-38ff8d0f77b1 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:20.066916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 126/249] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:19 2022] GET /identity/v3/role_assignments?user.id=b979812232ec49879fa487443941740e&scope.project.id=bbe948d5cac04aeb909127cea42f77b4&role.id=d43f8f61eb5f4489acfd5fba99b85919 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:20.068026 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a544938-ac4c-453b-bf13-c83d974db0d1 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:20.068392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a544938-ac4c-453b-bf13-c83d974db0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:20.069022 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a544938-ac4c-453b-bf13-c83d974db0d1 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:20.070015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 127/250] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:20.077764 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7a922-8262-42b8-8b9c-dcf0efda34b6 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:20.078224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7a922-8262-42b8-8b9c-dcf0efda34b6 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:20.078786 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7a922-8262-42b8-8b9c-dcf0efda34b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:20.102056 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:20.102673 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-e4b7a922-8262-42b8-8b9c-dcf0efda34b6 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:20.105432 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e4b7a922-8262-42b8-8b9c-dcf0efda34b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:20.112525 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243eb61-a6f8-4b10-81cf-ab6bd5f90230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:20.113264 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243eb61-a6f8-4b10-81cf-ab6bd5f90230 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:20.113545 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243eb61-a6f8-4b10-81cf-ab6bd5f90230 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:20.113819 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243eb61-a6f8-4b10-81cf-ab6bd5f90230 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:20.114589 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243eb61-a6f8-4b10-81cf-ab6bd5f90230 None admin] RBAC: Adding query filter params (name=member) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:20.114893 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243eb61-a6f8-4b10-81cf-ab6bd5f90230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:20.117036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243eb61-a6f8-4b10-81cf-ab6bd5f90230 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:20.135485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 124/251] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:20.141562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243eb61-a6f8-4b10-81cf-ab6bd5f90230 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:20.208663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 128/252] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:20.234197 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7a922-8262-42b8-8b9c-dcf0efda34b6 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:20.278813 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243eb61-a6f8-4b10-81cf-ab6bd5f90230 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:20.330988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cfb6c3-788e-48db-9c30-8a8dbc0c4952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:20.331890 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfb6c3-788e-48db-9c30-8a8dbc0c4952 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:20.332178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfb6c3-788e-48db-9c30-8a8dbc0c4952 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:20.332462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfb6c3-788e-48db-9c30-8a8dbc0c4952 None admin] PATH_INFO: `/v3/users/c8e616111b884178bca12f6877a18f82` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:20.334098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cfb6c3-788e-48db-9c30-8a8dbc0c4952 None admin] RBAC: Authorizing `identity:get_user(user_id=c8e616111b884178bca12f6877a18f82)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:20.335422 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cfb6c3-788e-48db-9c30-8a8dbc0c4952 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:20.337386 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 125/253] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:20 2022] GET /identity/v3/users/c8e616111b884178bca12f6877a18f82 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:20.338750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecb9a5-67cd-4bc5-9ae1-b5718f59b380 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:20.339042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecb9a5-67cd-4bc5-9ae1-b5718f59b380 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:20.339323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecb9a5-67cd-4bc5-9ae1-b5718f59b380 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:20.340595 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 126/254] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:20.343073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecb9a5-67cd-4bc5-9ae1-b5718f59b380 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:20.356757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7a922-8262-42b8-8b9c-dcf0efda34b6 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:20.358502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:20.396204 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fb133-5ea2-4d42-ba35-a016d5eb80f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:20.397216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fb133-5ea2-4d42-ba35-a016d5eb80f8 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:20.397626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fb133-5ea2-4d42-ba35-a016d5eb80f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:20.398151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fb133-5ea2-4d42-ba35-a016d5eb80f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:20.409943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fb133-5ea2-4d42-ba35-a016d5eb80f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:20.419112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-497fb133-5ea2-4d42-ba35-a016d5eb80f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:20.419112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:20.419112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:20.419112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:20.419112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:20.419112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:20.419112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:20.419112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:20.419112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:20.419112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:20.419112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:20.419112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:20.419112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:20.419112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:20.419112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:20.419112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:20.420183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:20.420183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:20.420183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:20.420183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:20.420183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:20.420183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:20.420183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:20.420183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:20.420183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:20.420183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:20.420183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:20.420183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:20.420183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:20.420183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:20.420183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:20.420183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:20.421089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:20.421089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:20.421089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:20.421089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:20.421089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:20.421089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:20.421089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:20.421089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:20.421089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:20.421089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:20.421089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:20.421089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:20.423130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 129/255] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:52:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:20.424062 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:20.424509 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a3ec6-fe2e-40a4-924d-39196138ab1c None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:20.429382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a3ec6-fe2e-40a4-924d-39196138ab1c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:20.429781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a3ec6-fe2e-40a4-924d-39196138ab1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:20.448940 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-ef0a3ec6-fe2e-40a4-924d-39196138ab1c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:20.450340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ef0a3ec6-fe2e-40a4-924d-39196138ab1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:20.476380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecb9a5-67cd-4bc5-9ae1-b5718f59b380 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:20.525689 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ad51b-1329-4f82-93cb-ed9f5f8cbc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:20.527948 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ad51b-1329-4f82-93cb-ed9f5f8cbc71 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:20.528159 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ad51b-1329-4f82-93cb-ed9f5f8cbc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:20.528385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ad51b-1329-4f82-93cb-ed9f5f8cbc71 None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:20.531855 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ad51b-1329-4f82-93cb-ed9f5f8cbc71 None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:20.534154 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ad51b-1329-4f82-93cb-ed9f5f8cbc71 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:20.539313 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 127/256] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:20 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:20.541067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ad51b-1329-4f82-93cb-ed9f5f8cbc71 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:20.550995 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 130/257] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:20.551930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a3ec6-fe2e-40a4-924d-39196138ab1c None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:20.731824 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ad51b-1329-4f82-93cb-ed9f5f8cbc71 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:20.745085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:20.759709 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a3ec6-fe2e-40a4-924d-39196138ab1c None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:20.788385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:20.802815 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418bbaf-df73-4024-b971-9b4043ed18e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:20.803492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418bbaf-df73-4024-b971-9b4043ed18e7 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:20.803776 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418bbaf-df73-4024-b971-9b4043ed18e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:20.804050 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418bbaf-df73-4024-b971-9b4043ed18e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:20.804828 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418bbaf-df73-4024-b971-9b4043ed18e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:20.805136 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418bbaf-df73-4024-b971-9b4043ed18e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:20.807799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418bbaf-df73-4024-b971-9b4043ed18e7 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:20.825622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 128/258] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:52:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:20.827977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418bbaf-df73-4024-b971-9b4043ed18e7 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:20.831010 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d5dcd-ad52-4c71-9a6a-ff9999362866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:20.831912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d5dcd-ad52-4c71-9a6a-ff9999362866 None admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:20.832193 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d5dcd-ad52-4c71-9a6a-ff9999362866 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:20.832465 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d5dcd-ad52-4c71-9a6a-ff9999362866 None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4/users/c8e616111b884178bca12f6877a18f82/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:20.836599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d5dcd-ad52-4c71-9a6a-ff9999362866 None admin] RBAC: Authorizing `identity:create_grant(project_id=bbe948d5cac04aeb909127cea42f77b4, user_id=c8e616111b884178bca12f6877a18f82, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:20.838613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d5dcd-ad52-4c71-9a6a-ff9999362866 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:20.862908 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 131/259] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:52:20 2022] PUT /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4/users/c8e616111b884178bca12f6877a18f82/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:52:20.865522 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d5dcd-ad52-4c71-9a6a-ff9999362866 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:21.045005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418bbaf-df73-4024-b971-9b4043ed18e7 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:21.106283 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a92415-e896-432c-84ec-9670cbfe08dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:21.107642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a92415-e896-432c-84ec-9670cbfe08dd None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:21.108844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a92415-e896-432c-84ec-9670cbfe08dd None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:21.109503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a92415-e896-432c-84ec-9670cbfe08dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:21.139742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a92415-e896-432c-84ec-9670cbfe08dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:21.148678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-31a92415-e896-432c-84ec-9670cbfe08dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:52:21.148678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:21.148678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:21.148678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:21.148678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:21.148678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:21.148678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:21.148678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:21.148678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:21.148678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:21.148678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:21.148678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:21.148678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:21.148678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:21.148678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:21.148678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:21.149641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:21.149641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:21.149641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:21.149641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:21.149641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:21.149641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:21.149641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:21.149641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:21.149641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:21.149641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:21.149641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:21.149641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:21.149641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:21.149641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:21.149641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:21.149641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:21.150992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:21.150992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:21.150992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:21.150992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:21.150992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:21.150992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:21.150992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:21.150992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:21.150992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:21.150992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:21.150992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:52:21.150992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:21.161176 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d5dcd-ad52-4c71-9a6a-ff9999362866 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:21.164595 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 129/260] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:52:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:21.170995 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a92415-e896-432c-84ec-9670cbfe08dd None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:21.232934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87418c-2e3b-415d-81d2-91b187f8b5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:21.235300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87418c-2e3b-415d-81d2-91b187f8b5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:21.237971 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87418c-2e3b-415d-81d2-91b187f8b5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:21.238516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87418c-2e3b-415d-81d2-91b187f8b5a6 None admin] PATH_INFO: `/v3/users/00c92ff7818146a5929c9078f3dcf87f` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:21.240054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87418c-2e3b-415d-81d2-91b187f8b5a6 None admin] RBAC: Authorizing `identity:get_user(user_id=00c92ff7818146a5929c9078f3dcf87f)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:21.242618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87418c-2e3b-415d-81d2-91b187f8b5a6 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:21.250364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 132/261] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:20 2022] GET /identity/v3/users/00c92ff7818146a5929c9078f3dcf87f => generated 283 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:21.255355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87418c-2e3b-415d-81d2-91b187f8b5a6 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:21.273503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:21.307201 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:21.401843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a92415-e896-432c-84ec-9670cbfe08dd None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:21.476516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87418c-2e3b-415d-81d2-91b187f8b5a6 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:21.489455 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de1969-80c8-4a5f-8d7f-8c515dfea67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:21.490144 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de1969-80c8-4a5f-8d7f-8c515dfea67e None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:21.490433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de1969-80c8-4a5f-8d7f-8c515dfea67e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:21.490717 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de1969-80c8-4a5f-8d7f-8c515dfea67e None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:21.491485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de1969-80c8-4a5f-8d7f-8c515dfea67e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:21.491796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de1969-80c8-4a5f-8d7f-8c515dfea67e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:21.495046 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de1969-80c8-4a5f-8d7f-8c515dfea67e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:21.514977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 130/262] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:21.520556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de1969-80c8-4a5f-8d7f-8c515dfea67e None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:21.545195 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ac144-b875-4c07-ac99-617380b26732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:21.546217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ac144-b875-4c07-ac99-617380b26732 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:21.546713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ac144-b875-4c07-ac99-617380b26732 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:21.547225 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ac144-b875-4c07-ac99-617380b26732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:21.690007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ac144-b875-4c07-ac99-617380b26732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:21.692788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ac144-b875-4c07-ac99-617380b26732 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:21.703928 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de1969-80c8-4a5f-8d7f-8c515dfea67e None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:21.728750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:21.735046 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:21.772627 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d3fa7-2598-4dae-a54c-32fdfd66f3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:21.773545 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d3fa7-2598-4dae-a54c-32fdfd66f3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:21.773829 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d3fa7-2598-4dae-a54c-32fdfd66f3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:21.774106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d3fa7-2598-4dae-a54c-32fdfd66f3b8 None admin] PATH_INFO: `/v3/users/00c92ff7818146a5929c9078f3dcf87f` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:21.775357 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d3fa7-2598-4dae-a54c-32fdfd66f3b8 None admin] RBAC: Authorizing `identity:get_user(user_id=00c92ff7818146a5929c9078f3dcf87f)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:21.777324 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d3fa7-2598-4dae-a54c-32fdfd66f3b8 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:21.779424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 131/263] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:21 2022] GET /identity/v3/users/00c92ff7818146a5929c9078f3dcf87f => generated 283 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:21.784954 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d3fa7-2598-4dae-a54c-32fdfd66f3b8 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:21.888362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 133/264] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 21:52:21.894004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ac144-b875-4c07-ac99-617380b26732 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:21.948424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d3fa7-2598-4dae-a54c-32fdfd66f3b8 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:22.000616 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f11a3-1c4c-48bc-a257-14f2b8c2a370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:22.001304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f11a3-1c4c-48bc-a257-14f2b8c2a370 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:22.001588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f11a3-1c4c-48bc-a257-14f2b8c2a370 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:22.001867 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f11a3-1c4c-48bc-a257-14f2b8c2a370 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:22.043609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ac144-b875-4c07-ac99-617380b26732 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:22.067048 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:22.081023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:22.101285 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f11a3-1c4c-48bc-a257-14f2b8c2a370 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:22.103306 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f11a3-1c4c-48bc-a257-14f2b8c2a370 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:22.112369 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbb1d4-851c-490d-b335-1eb580533985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:22.113369 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbb1d4-851c-490d-b335-1eb580533985 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:22.113649 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbb1d4-851c-490d-b335-1eb580533985 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:22.113922 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbb1d4-851c-490d-b335-1eb580533985 None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:22.117113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbb1d4-851c-490d-b335-1eb580533985 None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:22.119174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbb1d4-851c-490d-b335-1eb580533985 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:22.120972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 134/265] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:21 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:22.127167 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbb1d4-851c-490d-b335-1eb580533985 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:22.273934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbb1d4-851c-490d-b335-1eb580533985 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:22.315284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a2fb8-7dc2-4b60-be44-cec0884abd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:22.315284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a2fb8-7dc2-4b60-be44-cec0884abd87 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:22.315284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a2fb8-7dc2-4b60-be44-cec0884abd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:22.315284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a2fb8-7dc2-4b60-be44-cec0884abd87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:22.315705 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a2fb8-7dc2-4b60-be44-cec0884abd87 None admin] RBAC: Adding query filter params (role.id=160d4162340e472b9712a7f15cb2ecca, scope.project.id=bbe948d5cac04aeb909127cea42f77b4, user.id=00c92ff7818146a5929c9078f3dcf87f) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:22.315705 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a2fb8-7dc2-4b60-be44-cec0884abd87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:22.315705 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a2fb8-7dc2-4b60-be44-cec0884abd87 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:22.320027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 135/266] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:22 2022] GET /identity/v3/role_assignments?user.id=00c92ff7818146a5929c9078f3dcf87f&scope.project.id=bbe948d5cac04aeb909127cea42f77b4&role.id=160d4162340e472b9712a7f15cb2ecca => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:22.351619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 132/267] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 21:52:22.357568 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a2fb8-7dc2-4b60-be44-cec0884abd87 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:22.449147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:22.498418 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a2fb8-7dc2-4b60-be44-cec0884abd87 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:22.536099 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3395ae9-cfa5-4113-b5b0-73a98e71b5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:22.536919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3395ae9-cfa5-4113-b5b0-73a98e71b5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:22.537200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3395ae9-cfa5-4113-b5b0-73a98e71b5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:22.537480 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3395ae9-cfa5-4113-b5b0-73a98e71b5d9 None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:22.538694 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3395ae9-cfa5-4113-b5b0-73a98e71b5d9 None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:22.540149 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3395ae9-cfa5-4113-b5b0-73a98e71b5d9 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:22.542528 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 136/268] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:22 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:22.548491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f11a3-1c4c-48bc-a257-14f2b8c2a370 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:22.654934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:22.677466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f11a3-1c4c-48bc-a257-14f2b8c2a370 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:22.726880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8720a-a107-48e7-8e7d-36e48aa595cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:22.727607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8720a-a107-48e7-8e7d-36e48aa595cf None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:22.728470 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8720a-a107-48e7-8e7d-36e48aa595cf None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:22.728763 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8720a-a107-48e7-8e7d-36e48aa595cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:22.729614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8720a-a107-48e7-8e7d-36e48aa595cf None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.project.id=bbe948d5cac04aeb909127cea42f77b4, user.id=00c92ff7818146a5929c9078f3dcf87f) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:22.729924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8720a-a107-48e7-8e7d-36e48aa595cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:22.731515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8720a-a107-48e7-8e7d-36e48aa595cf None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:22.741664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 133/269] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:22 2022] GET /identity/v3/role_assignments?user.id=00c92ff7818146a5929c9078f3dcf87f&scope.project.id=bbe948d5cac04aeb909127cea42f77b4&role.id=053d1730ed764973abb1490894e07681 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:22.795597 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530481c9-7172-483e-b3e5-b8e1a114f0c6 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:22.796085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530481c9-7172-483e-b3e5-b8e1a114f0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:22.796334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530481c9-7172-483e-b3e5-b8e1a114f0c6 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:22.797843 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 137/270] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:22.803486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58cf45-3ded-46f6-b910-c43bebf957a5 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:22.803776 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58cf45-3ded-46f6-b910-c43bebf957a5 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:22.804060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58cf45-3ded-46f6-b910-c43bebf957a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:22.829209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-9c58cf45-3ded-46f6-b910-c43bebf957a5 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:22.830436 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-9c58cf45-3ded-46f6-b910-c43bebf957a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:22.925466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 134/271] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:22.971747 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530481c9-7172-483e-b3e5-b8e1a114f0c6 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:23.093969 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530481c9-7172-483e-b3e5-b8e1a114f0c6 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:23.117565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:23.130139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f4dd4-88ae-45bd-ad58-ff23e21ec778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:23.131394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f4dd4-88ae-45bd-ad58-ff23e21ec778 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:23.132115 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f4dd4-88ae-45bd-ad58-ff23e21ec778 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:23.132387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f4dd4-88ae-45bd-ad58-ff23e21ec778 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:23.144338 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f4dd4-88ae-45bd-ad58-ff23e21ec778 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:23.146938 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d06bfd-bd4c-41c6-a9b0-43c1067c52e9 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:23.147450 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d06bfd-bd4c-41c6-a9b0-43c1067c52e9 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:23.147930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d06bfd-bd4c-41c6-a9b0-43c1067c52e9 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:23.150430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 135/272] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:23.154331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-0a9f4dd4-88ae-45bd-ad58-ff23e21ec778 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:23.154331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:23.154331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:23.154331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:23.154331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:23.154331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:23.154331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:23.154331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:23.154331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:23.154331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:23.154331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:23.154331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:23.154331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:23.154331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:23.154331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:23.154331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:23.155213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:23.155213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:23.155213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:23.155213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:23.155213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:23.155213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:23.155213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:23.155213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:23.155213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:23.155213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:23.155213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:23.155213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:23.155213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:23.155213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:23.155213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:23.155213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:23.155991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:23.155991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:23.155991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:23.155991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:23.155991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:23.155991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:23.155991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:23.155991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:23.155991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:23.155991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:23.155991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:23.155991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:23.155991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 138/273] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:23.163146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d06bfd-bd4c-41c6-a9b0-43c1067c52e9 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:23.167301 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c421f4-9c5c-4b07-b0e8-9e2e6cab9674 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:23.167820 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c421f4-9c5c-4b07-b0e8-9e2e6cab9674 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:23.168246 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c421f4-9c5c-4b07-b0e8-9e2e6cab9674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:23.190581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-f6c421f4-9c5c-4b07-b0e8-9e2e6cab9674 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:23.192316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-f6c421f4-9c5c-4b07-b0e8-9e2e6cab9674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:23.276845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 139/274] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:23.285702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:23.306344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c421f4-9c5c-4b07-b0e8-9e2e6cab9674 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:23.321126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d06bfd-bd4c-41c6-a9b0-43c1067c52e9 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:23.366701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16bd140-698f-4ce4-8d5c-0b9a0eee3a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:23.368223 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bd140-698f-4ce4-8d5c-0b9a0eee3a65 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:23.369379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bd140-698f-4ce4-8d5c-0b9a0eee3a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:23.369676 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bd140-698f-4ce4-8d5c-0b9a0eee3a65 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:23.370449 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16bd140-698f-4ce4-8d5c-0b9a0eee3a65 None admin] RBAC: Adding query filter params (name=member) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:23.370763 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16bd140-698f-4ce4-8d5c-0b9a0eee3a65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:23.372175 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16bd140-698f-4ce4-8d5c-0b9a0eee3a65 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:23.380652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 136/275] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:23.386487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9064f-00bb-45f6-b015-cf8372dd406b None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:23.386779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9064f-00bb-45f6-b015-cf8372dd406b None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:23.387057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9064f-00bb-45f6-b015-cf8372dd406b None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:23.388533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 137/276] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:23.389098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9064f-00bb-45f6-b015-cf8372dd406b None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:23.424401 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c421f4-9c5c-4b07-b0e8-9e2e6cab9674 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:23.436538 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:23.460611 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f3551-2ee3-4853-b136-89a2b961b82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:23.461503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f3551-2ee3-4853-b136-89a2b961b82b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:23.461793 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f3551-2ee3-4853-b136-89a2b961b82b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:23.462075 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f3551-2ee3-4853-b136-89a2b961b82b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:23.463386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f3551-2ee3-4853-b136-89a2b961b82b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:23.464872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f3551-2ee3-4853-b136-89a2b961b82b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:23.466676 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 140/277] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:23.467642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e62db3-42e1-4b0d-8fb6-96ffe6e2f370 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:23.467924 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e62db3-42e1-4b0d-8fb6-96ffe6e2f370 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:23.468200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e62db3-42e1-4b0d-8fb6-96ffe6e2f370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:23.485159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-d4e62db3-42e1-4b0d-8fb6-96ffe6e2f370 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:23.486328 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d4e62db3-42e1-4b0d-8fb6-96ffe6e2f370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:23.510880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9064f-00bb-45f6-b015-cf8372dd406b None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:23.547285 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a00b6a-e9ef-4a06-9a01-dd5688458373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:23.548161 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a00b6a-e9ef-4a06-9a01-dd5688458373 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:23.548448 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a00b6a-e9ef-4a06-9a01-dd5688458373 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:23.548732 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a00b6a-e9ef-4a06-9a01-dd5688458373 None admin] PATH_INFO: `/v3/users/6b4f22313b4541c3bcc05dfc903d6c82` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:23.551628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:23.552597 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 141/278] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:23.557928 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e62db3-42e1-4b0d-8fb6-96ffe6e2f370 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:23.566455 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a00b6a-e9ef-4a06-9a01-dd5688458373 None admin] RBAC: Authorizing `identity:get_user(user_id=6b4f22313b4541c3bcc05dfc903d6c82)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:23.567952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a00b6a-e9ef-4a06-9a01-dd5688458373 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:23.569930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 138/279] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:23 2022] GET /identity/v3/users/6b4f22313b4541c3bcc05dfc903d6c82 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:23.570942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ef945-bd88-4ed2-9af9-30c06201b086 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:23.571230 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ef945-bd88-4ed2-9af9-30c06201b086 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:23.571594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ef945-bd88-4ed2-9af9-30c06201b086 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:23.573015 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 139/280] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:23.577844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ef945-bd88-4ed2-9af9-30c06201b086 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:23.660438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e62db3-42e1-4b0d-8fb6-96ffe6e2f370 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:23.670128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:23.681074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ef945-bd88-4ed2-9af9-30c06201b086 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:23.695025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baec441-de5e-41d8-b198-a63dfdeb6bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:23.695704 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baec441-de5e-41d8-b198-a63dfdeb6bc3 None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:23.695990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baec441-de5e-41d8-b198-a63dfdeb6bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:23.696265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baec441-de5e-41d8-b198-a63dfdeb6bc3 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:23.697284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baec441-de5e-41d8-b198-a63dfdeb6bc3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:23.698789 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baec441-de5e-41d8-b198-a63dfdeb6bc3 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:23.717447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c63fd-f432-465b-8cf5-1f34d236e49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:23.718701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c63fd-f432-465b-8cf5-1f34d236e49d None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:23.718701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c63fd-f432-465b-8cf5-1f34d236e49d None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:23.718918 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c63fd-f432-465b-8cf5-1f34d236e49d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:23.719703 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c63fd-f432-465b-8cf5-1f34d236e49d None admin] RBAC: Adding query filter params (role.id=b5ee21738ecb4079977d078d50ddbcf9, user.id=6b4f22313b4541c3bcc05dfc903d6c82, scope.system=all) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:23.719833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 142/281] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:52:23 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:52:23.720377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c63fd-f432-465b-8cf5-1f34d236e49d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:23.720660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baec441-de5e-41d8-b198-a63dfdeb6bc3 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:23.721920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c63fd-f432-465b-8cf5-1f34d236e49d None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:23.735836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 140/282] 10.222.0.44 () {56 vars in 1308 bytes} [Tue Jul 12 21:52:23 2022] GET /identity/v3/role_assignments?user.id=6b4f22313b4541c3bcc05dfc903d6c82&scope.system=all&role.id=b5ee21738ecb4079977d078d50ddbcf9 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:23.737624 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785e2cd-e66d-455c-9899-9a58562cfe19 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:23.737913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785e2cd-e66d-455c-9899-9a58562cfe19 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:23.738186 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785e2cd-e66d-455c-9899-9a58562cfe19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:23.760651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-4785e2cd-e66d-455c-9899-9a58562cfe19 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:23.762564 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4785e2cd-e66d-455c-9899-9a58562cfe19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:23.836751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:23.855256 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 141/283] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:23.874804 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baec441-de5e-41d8-b198-a63dfdeb6bc3 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:23.880685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785e2cd-e66d-455c-9899-9a58562cfe19 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:23.922050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc306b0-a43a-4659-962a-94b2d4eb39e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:23.922783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc306b0-a43a-4659-962a-94b2d4eb39e8 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:23.923061 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc306b0-a43a-4659-962a-94b2d4eb39e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:23.923334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc306b0-a43a-4659-962a-94b2d4eb39e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:23.935493 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc306b0-a43a-4659-962a-94b2d4eb39e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:23.948558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-9cc306b0-a43a-4659-962a-94b2d4eb39e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:23.948558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:23.948558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:23.948558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:23.948558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:23.948558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:23.948558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:23.948558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:23.948558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:23.948558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:23.948558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:23.948558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:23.948558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:23.948558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:23.948558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:23.948558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:23.949496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:23.949496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:23.949496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:23.949496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:23.949496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:23.949496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:23.949496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:23.949496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:23.949496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:23.949496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:23.949496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:23.949496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:23.949496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:23.949496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:23.949496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:23.949496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:23.950322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:23.950322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:23.950322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:23.950322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:23.950322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:23.950322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:23.950322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:23.950322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:23.950322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:23.950322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:23.950322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:23.950322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:23.950322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 143/284] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:23.956085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc306b0-a43a-4659-962a-94b2d4eb39e8 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:23.989746 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:24.034520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785e2cd-e66d-455c-9899-9a58562cfe19 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:24.081064 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d68b99-a018-4948-a685-d68f09263dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:24.081962 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d68b99-a018-4948-a685-d68f09263dbe None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:24.082321 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d68b99-a018-4948-a685-d68f09263dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:24.082667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d68b99-a018-4948-a685-d68f09263dbe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:24.097379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d68b99-a018-4948-a685-d68f09263dbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:24.097729 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc306b0-a43a-4659-962a-94b2d4eb39e8 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:24.105966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-c6d68b99-a018-4948-a685-d68f09263dbe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:24.105966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:24.105966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:24.105966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:24.105966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:24.105966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:24.105966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:24.105966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:24.105966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:24.105966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:24.105966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:24.105966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:24.105966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:24.105966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:24.105966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:24.105966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:24.106894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:24.106894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:24.106894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:24.106894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:24.106894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:24.106894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:24.106894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:24.106894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:24.106894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:24.106894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:24.106894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:24.106894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:24.106894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:24.106894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:24.106894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:24.106894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:24.107669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:24.107669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:24.107669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:24.107669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:24.107669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:24.107669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:24.107669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:24.107669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:24.107669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:24.107669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:24.107669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:24.107669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:24.111011 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 142/285] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:52:24.113956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d68b99-a018-4948-a685-d68f09263dbe None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:24.141210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ed44b-76a9-4203-ab0c-4d07c4b39ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:24.142763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ed44b-76a9-4203-ab0c-4d07c4b39ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:24.143050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ed44b-76a9-4203-ab0c-4d07c4b39ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:24.143331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ed44b-76a9-4203-ab0c-4d07c4b39ee0 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:24.146380 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ed44b-76a9-4203-ab0c-4d07c4b39ee0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:24.146380 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ed44b-76a9-4203-ab0c-4d07c4b39ee0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:24.146380 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ed44b-76a9-4203-ab0c-4d07c4b39ee0 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:24.155525 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 144/286] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:24.162779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ed44b-76a9-4203-ab0c-4d07c4b39ee0 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:24.192694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:24.274519 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:24.308168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d68b99-a018-4948-a685-d68f09263dbe None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:24.311547 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ed44b-76a9-4203-ab0c-4d07c4b39ee0 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:24.348350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997b56d-8294-4c6c-81ab-d5535909222f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:24.349057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997b56d-8294-4c6c-81ab-d5535909222f None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:24.349344 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997b56d-8294-4c6c-81ab-d5535909222f None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:24.349626 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997b56d-8294-4c6c-81ab-d5535909222f None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:24.350390 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997b56d-8294-4c6c-81ab-d5535909222f None admin] RBAC: Adding query filter params (name=member) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:24.350701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997b56d-8294-4c6c-81ab-d5535909222f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:24.352112 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997b56d-8294-4c6c-81ab-d5535909222f None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:24.362810 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 143/287] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:24.367977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997b56d-8294-4c6c-81ab-d5535909222f None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:24.401275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28369947-1ec8-48d3-b985-53c135349683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:24.402161 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28369947-1ec8-48d3-b985-53c135349683 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:24.402446 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28369947-1ec8-48d3-b985-53c135349683 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:24.402725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28369947-1ec8-48d3-b985-53c135349683 None admin] PATH_INFO: `/v3/users/b979812232ec49879fa487443941740e` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:24.404148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28369947-1ec8-48d3-b985-53c135349683 None admin] RBAC: Authorizing `identity:get_user(user_id=b979812232ec49879fa487443941740e)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:24.405725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28369947-1ec8-48d3-b985-53c135349683 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:24.408601 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 145/288] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:24 2022] GET /identity/v3/users/b979812232ec49879fa487443941740e => generated 297 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:24.418639 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28369947-1ec8-48d3-b985-53c135349683 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:24.539962 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997b56d-8294-4c6c-81ab-d5535909222f None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:24.565235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:24.585598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e76447-c57b-4889-8b8d-6fafc7254c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:24.586527 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e76447-c57b-4889-8b8d-6fafc7254c18 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:24.586815 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e76447-c57b-4889-8b8d-6fafc7254c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:24.587720 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e76447-c57b-4889-8b8d-6fafc7254c18 None admin] PATH_INFO: `/v3/users/c8e616111b884178bca12f6877a18f82` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:24.590344 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e76447-c57b-4889-8b8d-6fafc7254c18 None admin] RBAC: Authorizing `identity:get_user(user_id=c8e616111b884178bca12f6877a18f82)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:24.591892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e76447-c57b-4889-8b8d-6fafc7254c18 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:24.593641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 144/289] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:24 2022] GET /identity/v3/users/c8e616111b884178bca12f6877a18f82 => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:24.606047 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e76447-c57b-4889-8b8d-6fafc7254c18 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:24.651918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28369947-1ec8-48d3-b985-53c135349683 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:24.722381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2841c-4d66-4df2-9bb4-2e1337648ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:24.723264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2841c-4d66-4df2-9bb4-2e1337648ffe None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:24.723549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2841c-4d66-4df2-9bb4-2e1337648ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:24.723830 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2841c-4d66-4df2-9bb4-2e1337648ffe None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:24.725061 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2841c-4d66-4df2-9bb4-2e1337648ffe None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:24.726510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2841c-4d66-4df2-9bb4-2e1337648ffe None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:24.728564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 146/290] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:24 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:24.737674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2841c-4d66-4df2-9bb4-2e1337648ffe None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:24.750760 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e76447-c57b-4889-8b8d-6fafc7254c18 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:24.765089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:24.817995 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0ceff-92d3-4a77-9ca9-eab77e2f3c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:24.818576 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0ceff-92d3-4a77-9ca9-eab77e2f3c9e None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:24.818863 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0ceff-92d3-4a77-9ca9-eab77e2f3c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:24.819148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0ceff-92d3-4a77-9ca9-eab77e2f3c9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:24.862904 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2841c-4d66-4df2-9bb4-2e1337648ffe None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:24.873106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:24.900203 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b974e-0147-42f0-8e60-2d11d980cc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:24.901117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f0ceff-92d3-4a77-9ca9-eab77e2f3c9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:24.901249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b974e-0147-42f0-8e60-2d11d980cc24 None admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:24.901407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b974e-0147-42f0-8e60-2d11d980cc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:24.904730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b974e-0147-42f0-8e60-2d11d980cc24 None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4/users/b979812232ec49879fa487443941740e/roles/d43f8f61eb5f4489acfd5fba99b85919` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:24.904839 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f0ceff-92d3-4a77-9ca9-eab77e2f3c9e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:24.904911 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b974e-0147-42f0-8e60-2d11d980cc24 None admin] RBAC: Authorizing `identity:create_grant(project_id=bbe948d5cac04aeb909127cea42f77b4, user_id=b979812232ec49879fa487443941740e, role_id=d43f8f61eb5f4489acfd5fba99b85919)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:24.905204 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b974e-0147-42f0-8e60-2d11d980cc24 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:24.920507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 147/291] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:52:24 2022] PUT /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4/users/b979812232ec49879fa487443941740e/roles/d43f8f61eb5f4489acfd5fba99b85919 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:52:25.062357 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 145/292] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:25.068159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b974e-0147-42f0-8e60-2d11d980cc24 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:25.184029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:25.204179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b974e-0147-42f0-8e60-2d11d980cc24 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:25.246618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c02fd-1223-4856-a228-ddc33cda38fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:25.247557 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c02fd-1223-4856-a228-ddc33cda38fc None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:25.247841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c02fd-1223-4856-a228-ddc33cda38fc None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:25.248114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c02fd-1223-4856-a228-ddc33cda38fc None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:25.249411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c02fd-1223-4856-a228-ddc33cda38fc None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:25.250893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c02fd-1223-4856-a228-ddc33cda38fc None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:25.252614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 148/293] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:25 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:25.258541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0ceff-92d3-4a77-9ca9-eab77e2f3c9e None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:25.356683 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:25.384820 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0ceff-92d3-4a77-9ca9-eab77e2f3c9e None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:25.429478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc452878-8d6e-431a-8c4e-84e3e8fb2185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:25.430150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc452878-8d6e-431a-8c4e-84e3e8fb2185 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:25.430434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc452878-8d6e-431a-8c4e-84e3e8fb2185 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:25.430710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc452878-8d6e-431a-8c4e-84e3e8fb2185 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:25.432405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc452878-8d6e-431a-8c4e-84e3e8fb2185 None admin] RBAC: Adding query filter params (role.id=b5ee21738ecb4079977d078d50ddbcf9, scope.project.id=bbe948d5cac04aeb909127cea42f77b4, user.id=c8e616111b884178bca12f6877a18f82) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:25.432733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc452878-8d6e-431a-8c4e-84e3e8fb2185 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:25.434186 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc452878-8d6e-431a-8c4e-84e3e8fb2185 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:25.445818 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 146/294] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:25 2022] GET /identity/v3/role_assignments?user.id=c8e616111b884178bca12f6877a18f82&scope.project.id=bbe948d5cac04aeb909127cea42f77b4&role.id=b5ee21738ecb4079977d078d50ddbcf9 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:26.251035 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46ad19-6a32-4980-9af2-42d91487ec55 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:26.251526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46ad19-6a32-4980-9af2-42d91487ec55 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:26.251526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46ad19-6a32-4980-9af2-42d91487ec55 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:26.253147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 149/295] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:26.258356 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395778f-2b66-4761-b355-00e228cdb5fe None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:26.258801 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395778f-2b66-4761-b355-00e228cdb5fe None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:26.259553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395778f-2b66-4761-b355-00e228cdb5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:26.268965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61a74b-5a95-44ce-923f-e7c92bcbe589 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:26.269589 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61a74b-5a95-44ce-923f-e7c92bcbe589 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:26.270068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61a74b-5a95-44ce-923f-e7c92bcbe589 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:26.271902 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 150/296] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:52:26.277338 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b03b1-e1f0-46a1-a199-82090184ae97 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:26.277891 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b03b1-e1f0-46a1-a199-82090184ae97 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:26.278432 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b03b1-e1f0-46a1-a199-82090184ae97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:26.278782 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-4395778f-2b66-4761-b355-00e228cdb5fe None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:26.279975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4395778f-2b66-4761-b355-00e228cdb5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:26.296814 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-2d0b03b1-e1f0-46a1-a199-82090184ae97 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:26.298340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2d0b03b1-e1f0-46a1-a199-82090184ae97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:26.369302 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 147/297] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:26.388026 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 151/298] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:52:26.394934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395778f-2b66-4761-b355-00e228cdb5fe None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:26.411504 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d06b55-170e-489e-a59a-a73d23256b3d None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:26.411936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d06b55-170e-489e-a59a-a73d23256b3d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:26.412335 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d06b55-170e-489e-a59a-a73d23256b3d None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:26.414221 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 152/299] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:52:26.415073 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d06b55-170e-489e-a59a-a73d23256b3d None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:26.479313 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:26.508092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:26.517489 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395778f-2b66-4761-b355-00e228cdb5fe None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:26.529734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d06b55-170e-489e-a59a-a73d23256b3d None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:26.557065 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40361ad0-bed8-4e88-a272-d0e529a4665f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:26.557866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40361ad0-bed8-4e88-a272-d0e529a4665f None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:26.558418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40361ad0-bed8-4e88-a272-d0e529a4665f None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:26.558418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40361ad0-bed8-4e88-a272-d0e529a4665f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:26.566064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea9251-f6d4-46f6-a9b3-6de288219667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:26.567433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea9251-f6d4-46f6-a9b3-6de288219667 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:26.567784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea9251-f6d4-46f6-a9b3-6de288219667 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:26.567993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea9251-f6d4-46f6-a9b3-6de288219667 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:26.569164 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cea9251-f6d4-46f6-a9b3-6de288219667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:26.569382 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40361ad0-bed8-4e88-a272-d0e529a4665f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:26.570622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cea9251-f6d4-46f6-a9b3-6de288219667 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:26.573333 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 153/300] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:26.574265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10516ae6-2082-41bf-8b4a-5510608e3e08 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:26.574548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10516ae6-2082-41bf-8b4a-5510608e3e08 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:26.574821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10516ae6-2082-41bf-8b4a-5510608e3e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:26.582098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-40361ad0-bed8-4e88-a272-d0e529a4665f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:26.582098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:26.582098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:26.582098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:26.582098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:26.582098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:26.582098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:26.582098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:26.582098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:26.582098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:26.582098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:26.582098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:26.582098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:26.582098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:26.582098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:26.582098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:26.583034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:26.583034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:26.583034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:26.583034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:26.583034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:26.583034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:26.583034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:26.583034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:26.583034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:26.583034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:26.583034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:26.583034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:26.583034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:26.583034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:26.583034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:26.583034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:26.584384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:26.584384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:26.584384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:26.584384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:26.584384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:26.584384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:26.584384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:26.584384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:26.584384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:26.584384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:26.584384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:26.584384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:26.584384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 148/301] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:26.585352 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40361ad0-bed8-4e88-a272-d0e529a4665f None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:26.596322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-10516ae6-2082-41bf-8b4a-5510608e3e08 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:26.597536 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-10516ae6-2082-41bf-8b4a-5510608e3e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:26.675925 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 154/302] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:26.676799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10516ae6-2082-41bf-8b4a-5510608e3e08 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:26.702506 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40361ad0-bed8-4e88-a272-d0e529a4665f None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:26.742550 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:26.752209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063352e2-9452-405e-9904-765aa7270554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:26.752934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063352e2-9452-405e-9904-765aa7270554 None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:26.753231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063352e2-9452-405e-9904-765aa7270554 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:26.753506 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063352e2-9452-405e-9904-765aa7270554 None admin] PATH_INFO: `/v3/groups` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:26.754206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063352e2-9452-405e-9904-765aa7270554 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:26.755675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063352e2-9452-405e-9904-765aa7270554 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:26.765057 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:26.773083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 149/303] 10.222.0.44 () {60 vars in 1134 bytes} [Tue Jul 12 21:52:26 2022] POST /identity/v3/groups => generated 229 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:52:26.774237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063352e2-9452-405e-9904-765aa7270554 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:26.810401 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10516ae6-2082-41bf-8b4a-5510608e3e08 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:26.871330 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a289f-61c4-4ecc-bbdf-a03ccb25e157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:26.872148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a289f-61c4-4ecc-bbdf-a03ccb25e157 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:26.872498 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a289f-61c4-4ecc-bbdf-a03ccb25e157 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:26.872887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a289f-61c4-4ecc-bbdf-a03ccb25e157 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:26.874094 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a289f-61c4-4ecc-bbdf-a03ccb25e157 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:26.874489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a289f-61c4-4ecc-bbdf-a03ccb25e157 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:26.877062 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a289f-61c4-4ecc-bbdf-a03ccb25e157 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:26.891783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 155/304] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:26.897610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a289f-61c4-4ecc-bbdf-a03ccb25e157 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:26.915848 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063352e2-9452-405e-9904-765aa7270554 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:26.955800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4c99d-88e4-403e-9bef-c89b97be237e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:26.957061 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c99d-88e4-403e-9bef-c89b97be237e None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:26.960322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c99d-88e4-403e-9bef-c89b97be237e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:26.960851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c99d-88e4-403e-9bef-c89b97be237e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:26.986610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4c99d-88e4-403e-9bef-c89b97be237e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:26.995322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-54c4c99d-88e4-403e-9bef-c89b97be237e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:26.995322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:26.995322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:26.995322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:26.995322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:26.995322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:26.995322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:26.995322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:26.995322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:26.995322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:26.995322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:26.995322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:26.995322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:26.995322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:26.995322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:26.995322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:26.996430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:26.996430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:26.996430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:26.996430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:26.996430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:26.996430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:26.996430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:26.996430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:26.996430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:26.996430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:26.996430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:26.996430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:26.996430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:26.996430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:26.996430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:26.996430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:26.997556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:26.997556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:26.997556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:26.997556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:26.997556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:26.997556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:26.997556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:26.997556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:26.997556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:26.997556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:26.997556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:26.997556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:26.997556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 150/305] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:27.003382 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4c99d-88e4-403e-9bef-c89b97be237e None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:27.031848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a289f-61c4-4ecc-bbdf-a03ccb25e157 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:27.072012 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:27.091245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0d702-f0b5-4c18-9fde-290cc1edf472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:27.092225 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0d702-f0b5-4c18-9fde-290cc1edf472 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:27.092574 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0d702-f0b5-4c18-9fde-290cc1edf472 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:27.092971 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0d702-f0b5-4c18-9fde-290cc1edf472 None admin] PATH_INFO: `/v3/users/7e041055579f40199f9dc3ebc6c87eb2` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:27.106561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:27.112505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0d702-f0b5-4c18-9fde-290cc1edf472 None admin] RBAC: Authorizing `identity:get_user(user_id=7e041055579f40199f9dc3ebc6c87eb2)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:27.115050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0d702-f0b5-4c18-9fde-290cc1edf472 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:27.117699 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 156/306] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:26 2022] GET /identity/v3/users/7e041055579f40199f9dc3ebc6c87eb2 => generated 293 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:27.130234 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0d702-f0b5-4c18-9fde-290cc1edf472 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:27.133379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4c99d-88e4-403e-9bef-c89b97be237e None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:27.170962 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec02034-cf54-420d-aaae-ca574bc4eaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:27.171639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec02034-cf54-420d-aaae-ca574bc4eaa9 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:27.171938 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec02034-cf54-420d-aaae-ca574bc4eaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:27.172192 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec02034-cf54-420d-aaae-ca574bc4eaa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:27.172979 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec02034-cf54-420d-aaae-ca574bc4eaa9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:27.173289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec02034-cf54-420d-aaae-ca574bc4eaa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:27.174700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec02034-cf54-420d-aaae-ca574bc4eaa9 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:27.184947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 151/307] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:27.192289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec02034-cf54-420d-aaae-ca574bc4eaa9 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:27.255555 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0d702-f0b5-4c18-9fde-290cc1edf472 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:27.294494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5961973-52f1-449b-9fb9-70c37b563d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:27.295192 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5961973-52f1-449b-9fb9-70c37b563d66 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:27.295445 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5961973-52f1-449b-9fb9-70c37b563d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:27.295730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5961973-52f1-449b-9fb9-70c37b563d66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:27.296757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5961973-52f1-449b-9fb9-70c37b563d66 None admin] RBAC: Adding query filter params (role.id=d43f8f61eb5f4489acfd5fba99b85919, user.id=7e041055579f40199f9dc3ebc6c87eb2, scope.system=all) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:27.297010 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5961973-52f1-449b-9fb9-70c37b563d66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:27.298533 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5961973-52f1-449b-9fb9-70c37b563d66 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:27.305283 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 157/308] 10.222.0.44 () {56 vars in 1308 bytes} [Tue Jul 12 21:52:27 2022] GET /identity/v3/role_assignments?user.id=7e041055579f40199f9dc3ebc6c87eb2&scope.system=all&role.id=d43f8f61eb5f4489acfd5fba99b85919 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:27.306757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a80fd-5f52-4ed9-b09b-2def430b2b45 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:27.307057 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a80fd-5f52-4ed9-b09b-2def430b2b45 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:27.307378 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a80fd-5f52-4ed9-b09b-2def430b2b45 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:27.307633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec02034-cf54-420d-aaae-ca574bc4eaa9 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:27.308687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 158/309] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:27.318713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9b2ab-a4fe-4da7-b13a-3a2418d2772a None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:27.319332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9b2ab-a4fe-4da7-b13a-3a2418d2772a None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:27.319801 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9b2ab-a4fe-4da7-b13a-3a2418d2772a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:27.326290 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:27.340242 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-b6c9b2ab-a4fe-4da7-b13a-3a2418d2772a None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:27.341851 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b6c9b2ab-a4fe-4da7-b13a-3a2418d2772a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:27.351671 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884a5cc-bd8c-4b3a-a83f-e946ee4b42ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:27.352591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884a5cc-bd8c-4b3a-a83f-e946ee4b42ef None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:27.352926 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884a5cc-bd8c-4b3a-a83f-e946ee4b42ef None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:27.353211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884a5cc-bd8c-4b3a-a83f-e946ee4b42ef None admin] PATH_INFO: `/v3/users/6b4f22313b4541c3bcc05dfc903d6c82` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:27.354487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884a5cc-bd8c-4b3a-a83f-e946ee4b42ef None admin] RBAC: Authorizing `identity:get_user(user_id=6b4f22313b4541c3bcc05dfc903d6c82)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:27.355965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884a5cc-bd8c-4b3a-a83f-e946ee4b42ef None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:27.357874 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 152/310] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:27 2022] GET /identity/v3/users/6b4f22313b4541c3bcc05dfc903d6c82 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:27.364403 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884a5cc-bd8c-4b3a-a83f-e946ee4b42ef None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:27.422754 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:27.438738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 159/311] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:52:27.466965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9b2ab-a4fe-4da7-b13a-3a2418d2772a None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:27.520531 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884a5cc-bd8c-4b3a-a83f-e946ee4b42ef None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:27.563309 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a60c6-66a8-421a-8ded-74cc31ae6626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:27.564424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a60c6-66a8-421a-8ded-74cc31ae6626 None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:27.564424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a60c6-66a8-421a-8ded-74cc31ae6626 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:27.564867 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a60c6-66a8-421a-8ded-74cc31ae6626 None admin] PATH_INFO: `/v3/system/users/6b4f22313b4541c3bcc05dfc903d6c82/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:27.566678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a60c6-66a8-421a-8ded-74cc31ae6626 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6b4f22313b4541c3bcc05dfc903d6c82, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:27.568146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a60c6-66a8-421a-8ded-74cc31ae6626 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:27.580874 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 153/312] 10.222.0.44 () {58 vars in 1334 bytes} [Tue Jul 12 21:52:27 2022] PUT /identity/v3/system/users/6b4f22313b4541c3bcc05dfc903d6c82/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 21:52:27.595739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9b2ab-a4fe-4da7-b13a-3a2418d2772a None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:27.642318 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b58b01-56af-4624-a510-310323a05eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:27.644108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b58b01-56af-4624-a510-310323a05eea None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:27.644553 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b58b01-56af-4624-a510-310323a05eea None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:27.645017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b58b01-56af-4624-a510-310323a05eea None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:27.659534 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b58b01-56af-4624-a510-310323a05eea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:27.669257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-92b58b01-56af-4624-a510-310323a05eea None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:27.669257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:27.669257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:27.669257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:27.669257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:27.669257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:27.669257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:27.669257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:27.669257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:27.669257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:27.669257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:27.669257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:27.669257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:27.669257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:27.669257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:27.669257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:27.670396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:27.670396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:27.670396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:27.670396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:27.670396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:27.670396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:27.670396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:27.670396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:27.670396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:27.670396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:27.670396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:27.670396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:27.670396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:27.670396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:27.670396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:27.670396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:27.671367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:27.671367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:27.671367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:27.671367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:27.671367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:27.671367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:27.671367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:27.671367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:27.671367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:27.671367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:27.671367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:27.671367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:27.674031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 160/313] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:52:27.679823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a60c6-66a8-421a-8ded-74cc31ae6626 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:27.738417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:27.798627 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a60c6-66a8-421a-8ded-74cc31ae6626 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:27.840423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2faf5-e995-4ea4-901a-1d75c21c1927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:27.841086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2faf5-e995-4ea4-901a-1d75c21c1927 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:27.841376 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2faf5-e995-4ea4-901a-1d75c21c1927 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:27.841649 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2faf5-e995-4ea4-901a-1d75c21c1927 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:27.842400 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2faf5-e995-4ea4-901a-1d75c21c1927 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:27.842703 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2faf5-e995-4ea4-901a-1d75c21c1927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:27.844081 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2faf5-e995-4ea4-901a-1d75c21c1927 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:27.853649 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 154/314] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:27.858916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b58b01-56af-4624-a510-310323a05eea None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:27.914147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:27.977382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b58b01-56af-4624-a510-310323a05eea None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:28.015396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26576b7-279a-4c32-92a9-3ff3f98eb0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:28.016278 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26576b7-279a-4c32-92a9-3ff3f98eb0cb None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:28.016564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26576b7-279a-4c32-92a9-3ff3f98eb0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:28.016868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26576b7-279a-4c32-92a9-3ff3f98eb0cb None admin] PATH_INFO: `/v3/users/b979812232ec49879fa487443941740e` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:28.018083 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26576b7-279a-4c32-92a9-3ff3f98eb0cb None admin] RBAC: Authorizing `identity:get_user(user_id=b979812232ec49879fa487443941740e)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:28.019552 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26576b7-279a-4c32-92a9-3ff3f98eb0cb None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:28.021532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 161/315] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:27 2022] GET /identity/v3/users/b979812232ec49879fa487443941740e => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:28.029846 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2faf5-e995-4ea4-901a-1d75c21c1927 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:28.146565 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2faf5-e995-4ea4-901a-1d75c21c1927 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:28.156251 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:28.182587 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29500bb-4b62-4c2d-9977-b2510eeb5081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:28.183245 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29500bb-4b62-4c2d-9977-b2510eeb5081 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:28.183526 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29500bb-4b62-4c2d-9977-b2510eeb5081 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:28.183799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29500bb-4b62-4c2d-9977-b2510eeb5081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:28.253694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29500bb-4b62-4c2d-9977-b2510eeb5081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:28.255150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29500bb-4b62-4c2d-9977-b2510eeb5081 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:28.356225 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 155/316] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:28.362047 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26576b7-279a-4c32-92a9-3ff3f98eb0cb None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:28.473737 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26576b7-279a-4c32-92a9-3ff3f98eb0cb None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:28.495965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:28.511211 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923787b0-60a1-4ba7-9e37-1ff5264c36d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:28.512160 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923787b0-60a1-4ba7-9e37-1ff5264c36d1 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:28.512441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923787b0-60a1-4ba7-9e37-1ff5264c36d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:28.512777 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923787b0-60a1-4ba7-9e37-1ff5264c36d1 None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:28.514417 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923787b0-60a1-4ba7-9e37-1ff5264c36d1 None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:28.516315 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923787b0-60a1-4ba7-9e37-1ff5264c36d1 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:28.518066 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 162/317] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:28 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:28.525834 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29500bb-4b62-4c2d-9977-b2510eeb5081 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:28.541196 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:28.639750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29500bb-4b62-4c2d-9977-b2510eeb5081 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:28.674369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c496153-9963-4ba6-95c3-4cea76925a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:28.675032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c496153-9963-4ba6-95c3-4cea76925a93 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:28.675313 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c496153-9963-4ba6-95c3-4cea76925a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:28.675586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c496153-9963-4ba6-95c3-4cea76925a93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:28.676420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c496153-9963-4ba6-95c3-4cea76925a93 None admin] RBAC: Adding query filter params (role.id=d43f8f61eb5f4489acfd5fba99b85919, scope.project.id=bbe948d5cac04aeb909127cea42f77b4, user.id=b979812232ec49879fa487443941740e) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:28.676766 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c496153-9963-4ba6-95c3-4cea76925a93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:28.678306 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c496153-9963-4ba6-95c3-4cea76925a93 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:28.685093 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 156/318] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:28 2022] GET /identity/v3/role_assignments?user.id=b979812232ec49879fa487443941740e&scope.project.id=bbe948d5cac04aeb909127cea42f77b4&role.id=d43f8f61eb5f4489acfd5fba99b85919 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:29.017982 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63d06a-33b9-4b73-b0ce-160f49696da5 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:29.018318 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63d06a-33b9-4b73-b0ce-160f49696da5 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:29.018537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63d06a-33b9-4b73-b0ce-160f49696da5 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:29.019912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 163/319] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:29.025698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edfd93-2295-4755-b696-468851e3bf49 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:29.026059 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edfd93-2295-4755-b696-468851e3bf49 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:29.026346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edfd93-2295-4755-b696-468851e3bf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:29.042973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-a0edfd93-2295-4755-b696-468851e3bf49 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:29.044171 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a0edfd93-2295-4755-b696-468851e3bf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:29.110409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 157/320] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:29.134084 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63d06a-33b9-4b73-b0ce-160f49696da5 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:29.239463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63d06a-33b9-4b73-b0ce-160f49696da5 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:29.279355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65688734-ed5f-41bd-9bb8-3995fa0ae64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:29.280157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65688734-ed5f-41bd-9bb8-3995fa0ae64d None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:29.280492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65688734-ed5f-41bd-9bb8-3995fa0ae64d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:29.280799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65688734-ed5f-41bd-9bb8-3995fa0ae64d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:29.281990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65688734-ed5f-41bd-9bb8-3995fa0ae64d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:29.283442 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65688734-ed5f-41bd-9bb8-3995fa0ae64d None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:29.285315 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 164/321] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:29.290838 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edfd93-2295-4755-b696-468851e3bf49 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:29.293050 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:29.394857 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edfd93-2295-4755-b696-468851e3bf49 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:29.429671 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe726c-50b6-4cfb-a1cf-04767c931b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:29.430276 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe726c-50b6-4cfb-a1cf-04767c931b39 None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:29.430561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe726c-50b6-4cfb-a1cf-04767c931b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:29.430960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe726c-50b6-4cfb-a1cf-04767c931b39 None admin] PATH_INFO: `/v3/groups` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:29.431948 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe726c-50b6-4cfb-a1cf-04767c931b39 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:29.433434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe726c-50b6-4cfb-a1cf-04767c931b39 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:29.448615 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 158/322] 10.222.0.44 () {60 vars in 1134 bytes} [Tue Jul 12 21:52:29 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:52:29.484513 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86ff18-4cfa-48fa-982f-f4f845a52aec None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:29.485069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86ff18-4cfa-48fa-982f-f4f845a52aec None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:29.485141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86ff18-4cfa-48fa-982f-f4f845a52aec None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:29.487166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 165/323] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:29.492941 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a8f3b-aa61-411b-a40a-65a7f4b6f2e5 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:29.493193 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a8f3b-aa61-411b-a40a-65a7f4b6f2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:29.493462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a8f3b-aa61-411b-a40a-65a7f4b6f2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:29.511851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-829a8f3b-aa61-411b-a40a-65a7f4b6f2e5 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:29.513115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-829a8f3b-aa61-411b-a40a-65a7f4b6f2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:29.577982 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:29.595882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 159/324] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:29.618716 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86ff18-4cfa-48fa-982f-f4f845a52aec None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:29.684969 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:29.737867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86ff18-4cfa-48fa-982f-f4f845a52aec None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:29.775483 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573cffd-f4a7-4a53-81a2-04563e94c8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:29.776766 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573cffd-f4a7-4a53-81a2-04563e94c8ee None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:29.777270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573cffd-f4a7-4a53-81a2-04563e94c8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:29.777712 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573cffd-f4a7-4a53-81a2-04563e94c8ee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:29.790093 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573cffd-f4a7-4a53-81a2-04563e94c8ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:29.792733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c2ce3-6e0c-4770-9973-c45c3efd18f7 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:29.793044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c2ce3-6e0c-4770-9973-c45c3efd18f7 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:29.793327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c2ce3-6e0c-4770-9973-c45c3efd18f7 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:29.794843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 160/325] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:29.799270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-e573cffd-f4a7-4a53-81a2-04563e94c8ee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:29.799270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:29.799270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:29.799270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:29.799270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:29.799270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:29.799270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:29.799270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:29.799270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:29.799270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:29.799270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:29.799270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:29.799270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:29.799270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:29.799270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:29.799270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:29.800140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:29.800140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:29.800140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:29.800140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:29.800140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:29.800140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:29.800140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:29.800140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:29.800140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:29.800140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:29.800140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:29.800140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:29.800140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:29.800140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:29.800140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:29.800140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:29.800954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:29.800954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:29.800954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:29.800954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:29.800954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:29.800954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:29.800954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:29.800954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:29.800954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:29.800954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:29.800954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:29.800954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:29.800954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 166/326] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:29.801605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed95d0-2c52-44fa-b4e3-8618d8ded14f None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:29.802179 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed95d0-2c52-44fa-b4e3-8618d8ded14f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:29.802179 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed95d0-2c52-44fa-b4e3-8618d8ded14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:29.806999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573cffd-f4a7-4a53-81a2-04563e94c8ee None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:29.824883 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-68ed95d0-2c52-44fa-b4e3-8618d8ded14f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:29.826379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-68ed95d0-2c52-44fa-b4e3-8618d8ded14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:29.913751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 161/327] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:52:29.925905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573cffd-f4a7-4a53-81a2-04563e94c8ee None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:29.939057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed95d0-2c52-44fa-b4e3-8618d8ded14f None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:29.946400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:29.971995 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e01c0f-a451-4151-a18f-e1d0cb6a6698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:29.972752 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e01c0f-a451-4151-a18f-e1d0cb6a6698 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:29.972975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e01c0f-a451-4151-a18f-e1d0cb6a6698 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:29.973239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e01c0f-a451-4151-a18f-e1d0cb6a6698 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:29.974249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e01c0f-a451-4151-a18f-e1d0cb6a6698 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:29.974554 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e01c0f-a451-4151-a18f-e1d0cb6a6698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:29.978386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e01c0f-a451-4151-a18f-e1d0cb6a6698 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:29.988756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 167/328] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:29.996575 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e01c0f-a451-4151-a18f-e1d0cb6a6698 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:30.085585 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed95d0-2c52-44fa-b4e3-8618d8ded14f None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:30.117190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:30.131087 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e01c0f-a451-4151-a18f-e1d0cb6a6698 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:30.134432 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29ff8f-6912-4a45-8ec2-0217e956e021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:30.137729 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29ff8f-6912-4a45-8ec2-0217e956e021 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:30.138170 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29ff8f-6912-4a45-8ec2-0217e956e021 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:30.138590 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29ff8f-6912-4a45-8ec2-0217e956e021 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:30.154199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29ff8f-6912-4a45-8ec2-0217e956e021 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:30.164178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-0f29ff8f-6912-4a45-8ec2-0217e956e021 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:30.164178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:30.164178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:30.164178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:30.164178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:30.164178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:30.164178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:30.164178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:30.164178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:30.164178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:30.164178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:30.164178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:30.164178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:30.164178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:30.164178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:30.164178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:30.165493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:30.165493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:30.165493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:30.165493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:30.165493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:30.165493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:30.165493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:30.165493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:30.165493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:30.165493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:30.165493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:30.165493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:30.165493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:30.165493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:30.165493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:30.165493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:30.166465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:30.166465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:30.166465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:30.166465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:30.166465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:30.166465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:30.166465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:30.166465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:30.166465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:30.166465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:30.166465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:30.166465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:30.169310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 162/329] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:52:30.174972 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29ff8f-6912-4a45-8ec2-0217e956e021 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:30.187028 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dfb0f-bb60-47d0-8ddf-44bda519d556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:30.188562 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dfb0f-bb60-47d0-8ddf-44bda519d556 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:30.189033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dfb0f-bb60-47d0-8ddf-44bda519d556 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:30.189458 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dfb0f-bb60-47d0-8ddf-44bda519d556 None admin] PATH_INFO: `/v3/users/7e041055579f40199f9dc3ebc6c87eb2` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:30.191143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66dfb0f-bb60-47d0-8ddf-44bda519d556 None admin] RBAC: Authorizing `identity:get_user(user_id=7e041055579f40199f9dc3ebc6c87eb2)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:30.193675 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66dfb0f-bb60-47d0-8ddf-44bda519d556 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:30.196828 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 168/330] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:29 2022] GET /identity/v3/users/7e041055579f40199f9dc3ebc6c87eb2 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:30.203664 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dfb0f-bb60-47d0-8ddf-44bda519d556 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:30.261521 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:30.311877 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29ff8f-6912-4a45-8ec2-0217e956e021 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:30.338454 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dfb0f-bb60-47d0-8ddf-44bda519d556 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:30.355950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d33bac-ee4f-444c-80ad-911ecf065831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:30.356384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d33bac-ee4f-444c-80ad-911ecf065831 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:30.356679 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d33bac-ee4f-444c-80ad-911ecf065831 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:30.356999 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d33bac-ee4f-444c-80ad-911ecf065831 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:30.357931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d33bac-ee4f-444c-80ad-911ecf065831 None admin] RBAC: Adding query filter params (name=member) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:30.358353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d33bac-ee4f-444c-80ad-911ecf065831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:30.359954 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d33bac-ee4f-444c-80ad-911ecf065831 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:30.372059 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 163/331] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:30.378658 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d33bac-ee4f-444c-80ad-911ecf065831 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:30.381341 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7c76e-b9a8-44f2-bfcf-b1f8c24beebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:30.382785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7c76e-b9a8-44f2-bfcf-b1f8c24beebb None admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:30.382896 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7c76e-b9a8-44f2-bfcf-b1f8c24beebb None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:30.383143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7c76e-b9a8-44f2-bfcf-b1f8c24beebb None admin] PATH_INFO: `/v3/system/users/7e041055579f40199f9dc3ebc6c87eb2/roles/d43f8f61eb5f4489acfd5fba99b85919` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:30.385856 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7c76e-b9a8-44f2-bfcf-b1f8c24beebb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7e041055579f40199f9dc3ebc6c87eb2, role_id=d43f8f61eb5f4489acfd5fba99b85919)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:30.387311 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7c76e-b9a8-44f2-bfcf-b1f8c24beebb None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:30.400403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 169/332] 10.222.0.44 () {58 vars in 1334 bytes} [Tue Jul 12 21:52:30 2022] PUT /identity/v3/system/users/7e041055579f40199f9dc3ebc6c87eb2/roles/d43f8f61eb5f4489acfd5fba99b85919 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:52:30.440238 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:30.518402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d33bac-ee4f-444c-80ad-911ecf065831 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:30.560531 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eea10f-cd13-45c6-ab56-ab464bcafa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:30.561810 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eea10f-cd13-45c6-ab56-ab464bcafa88 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:30.562521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eea10f-cd13-45c6-ab56-ab464bcafa88 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:30.563299 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eea10f-cd13-45c6-ab56-ab464bcafa88 None admin] PATH_INFO: `/v3/users/6b4f22313b4541c3bcc05dfc903d6c82` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:30.566973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eea10f-cd13-45c6-ab56-ab464bcafa88 None admin] RBAC: Authorizing `identity:get_user(user_id=6b4f22313b4541c3bcc05dfc903d6c82)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:30.568673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eea10f-cd13-45c6-ab56-ab464bcafa88 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:30.571170 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 164/333] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:30 2022] GET /identity/v3/users/6b4f22313b4541c3bcc05dfc903d6c82 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:30.578555 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7c76e-b9a8-44f2-bfcf-b1f8c24beebb None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:30.722349 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7c76e-b9a8-44f2-bfcf-b1f8c24beebb None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:30.734406 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:30.766364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034977ae-8966-4e2d-aba6-0644d18b04f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:30.767052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034977ae-8966-4e2d-aba6-0644d18b04f4 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:30.767342 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034977ae-8966-4e2d-aba6-0644d18b04f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:30.767627 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034977ae-8966-4e2d-aba6-0644d18b04f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:30.768511 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034977ae-8966-4e2d-aba6-0644d18b04f4 None admin] RBAC: Adding query filter params (role.id=b5ee21738ecb4079977d078d50ddbcf9, user.id=6b4f22313b4541c3bcc05dfc903d6c82, scope.system=all) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:30.768846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034977ae-8966-4e2d-aba6-0644d18b04f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:30.770311 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034977ae-8966-4e2d-aba6-0644d18b04f4 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:30.784576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 170/334] 10.222.0.44 () {56 vars in 1308 bytes} [Tue Jul 12 21:52:30 2022] GET /identity/v3/role_assignments?user.id=6b4f22313b4541c3bcc05dfc903d6c82&scope.system=all&role.id=b5ee21738ecb4079977d078d50ddbcf9 => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:32.367691 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da273ef-6d49-46d5-a137-cc092736f686 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:32.368943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da273ef-6d49-46d5-a137-cc092736f686 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:32.369454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da273ef-6d49-46d5-a137-cc092736f686 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:32.371330 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 165/335] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:32.376800 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b2dbb-6ca6-40c7-8040-3862bec0555d None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:32.377257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b2dbb-6ca6-40c7-8040-3862bec0555d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:32.377441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b2dbb-6ca6-40c7-8040-3862bec0555d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:32.402030 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-4a4b2dbb-6ca6-40c7-8040-3862bec0555d None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:32.403273 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-4a4b2dbb-6ca6-40c7-8040-3862bec0555d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:32.485652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 171/336] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:32.534285 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da273ef-6d49-46d5-a137-cc092736f686 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:32.556735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b647e2-335b-4a30-9607-e7281e12197c None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:32.557173 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b647e2-335b-4a30-9607-e7281e12197c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:32.557975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b647e2-335b-4a30-9607-e7281e12197c None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:32.561426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 172/337] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:32.576331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ba42f-e51a-4100-b49c-317f8660a19d None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:32.576640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ba42f-e51a-4100-b49c-317f8660a19d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:32.577443 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ba42f-e51a-4100-b49c-317f8660a19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:32.610907 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-cc8ba42f-e51a-4100-b49c-317f8660a19d None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:32.612119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-cc8ba42f-e51a-4100-b49c-317f8660a19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:32.735316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da273ef-6d49-46d5-a137-cc092736f686 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:32.752820 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 173/338] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:52:32.761083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:32.790738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8573adb-38b3-44ba-9c8b-6b87afce1786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:32.791562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8573adb-38b3-44ba-9c8b-6b87afce1786 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:32.791849 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8573adb-38b3-44ba-9c8b-6b87afce1786 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:32.792134 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8573adb-38b3-44ba-9c8b-6b87afce1786 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:32.803267 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8573adb-38b3-44ba-9c8b-6b87afce1786 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:32.816028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-d8573adb-38b3-44ba-9c8b-6b87afce1786 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:32.816028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:32.816028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:32.816028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:32.816028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:32.816028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:32.816028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:32.816028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:32.816028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:32.816028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:32.816028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:32.816028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:32.816028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:32.816028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:32.816028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:32.816028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:32.817268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:32.817268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:32.817268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:32.817268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:32.817268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:32.817268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:32.817268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:32.817268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:32.817268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:32.817268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:32.817268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:32.817268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:32.817268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:32.817268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:32.817268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:32.817268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:32.818057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:32.818057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:32.818057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:32.818057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:32.818057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:32.818057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:32.818057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:32.818057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:32.818057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:32.818057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:32.818057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:32.818057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:32.818057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 166/339] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:52:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:32.823905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ba42f-e51a-4100-b49c-317f8660a19d None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:32.853950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:32.870658 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8573adb-38b3-44ba-9c8b-6b87afce1786 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:33.024951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ba42f-e51a-4100-b49c-317f8660a19d None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:33.087555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8573adb-38b3-44ba-9c8b-6b87afce1786 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:33.092269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1270b1-3549-4480-8898-68fa9c2e1672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:33.093277 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1270b1-3549-4480-8898-68fa9c2e1672 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:33.093755 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1270b1-3549-4480-8898-68fa9c2e1672 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:33.094221 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1270b1-3549-4480-8898-68fa9c2e1672 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:33.095162 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1270b1-3549-4480-8898-68fa9c2e1672 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:33.095666 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1270b1-3549-4480-8898-68fa9c2e1672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:33.097313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1270b1-3549-4480-8898-68fa9c2e1672 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:33.107525 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 174/340] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:52:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:33.109231 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3db713-0a31-46e8-be54-159628012461 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:33.109717 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3db713-0a31-46e8-be54-159628012461 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:33.110183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3db713-0a31-46e8-be54-159628012461 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:33.111647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 175/341] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:33.120144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3db713-0a31-46e8-be54-159628012461 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:33.135145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fdb10-795e-4ebb-b24b-d8f08c9bf5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:33.135145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fdb10-795e-4ebb-b24b-d8f08c9bf5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:33.135145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fdb10-795e-4ebb-b24b-d8f08c9bf5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:33.135145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fdb10-795e-4ebb-b24b-d8f08c9bf5b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:33.144474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501fdb10-795e-4ebb-b24b-d8f08c9bf5b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:33.152458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-501fdb10-795e-4ebb-b24b-d8f08c9bf5b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:33.152458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:33.152458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:33.152458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:33.152458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:33.152458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:33.152458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:33.152458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:33.152458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:33.152458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:33.152458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:33.152458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:33.152458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:33.152458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:33.152458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:33.152458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:33.153405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:33.153405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:33.153405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:33.153405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:33.153405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:33.153405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:33.153405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:33.153405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:33.153405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:33.153405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:33.153405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:33.153405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:33.153405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:33.153405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:33.153405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:33.153405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:33.154203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:33.154203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:33.154203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:33.154203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:33.154203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:33.154203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:33.154203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:33.154203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:33.154203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:33.154203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:33.154203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:33.154203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:33.160986 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 167/342] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:52:33.162568 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63076d-a25a-4c48-9034-8362f0653637 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:33.163102 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63076d-a25a-4c48-9034-8362f0653637 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:33.163585 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63076d-a25a-4c48-9034-8362f0653637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:33.182446 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-5f63076d-a25a-4c48-9034-8362f0653637 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:33.184000 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-5f63076d-a25a-4c48-9034-8362f0653637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:33.214401 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:33.245664 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:33.258215 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3db713-0a31-46e8-be54-159628012461 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:33.258417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 168/343] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:33.260114 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63076d-a25a-4c48-9034-8362f0653637 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:33.297589 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bec85f-cc61-4f37-b160-edb53f43c005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:33.298518 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bec85f-cc61-4f37-b160-edb53f43c005 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:33.298984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bec85f-cc61-4f37-b160-edb53f43c005 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:33.299464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bec85f-cc61-4f37-b160-edb53f43c005 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:33.370302 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63076d-a25a-4c48-9034-8362f0653637 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:33.379297 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bec85f-cc61-4f37-b160-edb53f43c005 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:33.381401 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bec85f-cc61-4f37-b160-edb53f43c005 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:33.407361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666047e0-04ef-4cd4-bd2b-02daaed6360d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:33.408459 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666047e0-04ef-4cd4-bd2b-02daaed6360d None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:33.408459 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666047e0-04ef-4cd4-bd2b-02daaed6360d None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:33.408957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666047e0-04ef-4cd4-bd2b-02daaed6360d None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:33.409850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666047e0-04ef-4cd4-bd2b-02daaed6360d None admin] RBAC: Adding query filter params (name=member) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:33.410215 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666047e0-04ef-4cd4-bd2b-02daaed6360d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:33.411727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666047e0-04ef-4cd4-bd2b-02daaed6360d None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:33.424993 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 169/344] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:33.426030 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666047e0-04ef-4cd4-bd2b-02daaed6360d None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:33.454412 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:33.498982 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 176/345] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 21:52:33.500601 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09cbc6-5f8d-4911-852d-194fa1925371 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:33.501235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09cbc6-5f8d-4911-852d-194fa1925371 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:33.501695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09cbc6-5f8d-4911-852d-194fa1925371 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:33.503389 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 177/346] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:33.506156 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:33.506732 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09cbc6-5f8d-4911-852d-194fa1925371 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:33.544056 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666047e0-04ef-4cd4-bd2b-02daaed6360d None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:33.582213 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33002a-5e38-4527-8c22-acb5dbd47393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:33.582971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33002a-5e38-4527-8c22-acb5dbd47393 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:33.583320 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33002a-5e38-4527-8c22-acb5dbd47393 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:33.583602 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33002a-5e38-4527-8c22-acb5dbd47393 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:33.596545 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33002a-5e38-4527-8c22-acb5dbd47393 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:33.606704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-1d33002a-5e38-4527-8c22-acb5dbd47393 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:33.606704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:33.606704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:33.606704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:33.606704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:33.606704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:33.606704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:33.606704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:33.606704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:33.606704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:33.606704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:33.606704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:33.606704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:33.606704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:33.606704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:33.606704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:33.607667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:33.607667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:33.607667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:33.607667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:33.607667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:33.607667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:33.607667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:33.607667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:33.607667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:33.607667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:33.607667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:33.607667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:33.607667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:33.607667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:33.607667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:33.607667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:33.608477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:33.608477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:33.608477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:33.608477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:33.608477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:33.608477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:33.608477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:33.608477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:33.608477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:33.608477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:33.608477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 21:52:33.608477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:33.608477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 170/347] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:33.609185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33002a-5e38-4527-8c22-acb5dbd47393 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:33.628733 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09cbc6-5f8d-4911-852d-194fa1925371 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:33.669445 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51773497-1a9b-4d65-8cf1-c4f9f66383d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:33.670567 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51773497-1a9b-4d65-8cf1-c4f9f66383d8 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:33.671043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51773497-1a9b-4d65-8cf1-c4f9f66383d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:33.671547 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51773497-1a9b-4d65-8cf1-c4f9f66383d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:33.702331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:33.723724 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33002a-5e38-4527-8c22-acb5dbd47393 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:33.741034 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:33.754394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51773497-1a9b-4d65-8cf1-c4f9f66383d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:33.756347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51773497-1a9b-4d65-8cf1-c4f9f66383d8 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:33.760937 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43121a27-8fb3-47d1-8b91-1f2c27854ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:33.761809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43121a27-8fb3-47d1-8b91-1f2c27854ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:33.762100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43121a27-8fb3-47d1-8b91-1f2c27854ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:33.762378 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43121a27-8fb3-47d1-8b91-1f2c27854ea4 None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:33.763592 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43121a27-8fb3-47d1-8b91-1f2c27854ea4 None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:33.765158 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43121a27-8fb3-47d1-8b91-1f2c27854ea4 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:33.767335 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 171/348] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:33 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:33.768507 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ced95-b388-4672-8f82-9782585b41f6 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:33.768833 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ced95-b388-4672-8f82-9782585b41f6 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:33.769117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ced95-b388-4672-8f82-9782585b41f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:33.787882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-2d2ced95-b388-4672-8f82-9782585b41f6 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:33.789272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-2d2ced95-b388-4672-8f82-9782585b41f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:33.865676 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 172/349] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:33.867733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ced95-b388-4672-8f82-9782585b41f6 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:33.875874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 178/350] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:33.877027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51773497-1a9b-4d65-8cf1-c4f9f66383d8 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:33.974973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:33.979589 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:33.991793 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51773497-1a9b-4d65-8cf1-c4f9f66383d8 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:33.996406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ced95-b388-4672-8f82-9782585b41f6 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:34.031115 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcc02b-29b0-4c49-8b5f-9ba844e93fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:34.032314 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcc02b-29b0-4c49-8b5f-9ba844e93fae None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:34.032874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcc02b-29b0-4c49-8b5f-9ba844e93fae None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:34.033359 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcc02b-29b0-4c49-8b5f-9ba844e93fae None admin] PATH_INFO: `/v3/groups/7716588d7cfa485b815c38966d72edf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:34.034702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af4225-5d81-40ed-b476-b69cf9a477a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:34.035383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af4225-5d81-40ed-b476-b69cf9a477a9 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:34.035672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af4225-5d81-40ed-b476-b69cf9a477a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:34.035950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af4225-5d81-40ed-b476-b69cf9a477a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:34.036863 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af4225-5d81-40ed-b476-b69cf9a477a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:34.037177 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af4225-5d81-40ed-b476-b69cf9a477a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:34.038918 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af4225-5d81-40ed-b476-b69cf9a477a9 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:34.042390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bcc02b-29b0-4c49-8b5f-9ba844e93fae None admin] RBAC: Authorizing `identity:get_group(group_id=7716588d7cfa485b815c38966d72edf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:34.044103 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bcc02b-29b0-4c49-8b5f-9ba844e93fae None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:34.046725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 179/351] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Jul 12 21:52:33 2022] GET /identity/v3/groups/7716588d7cfa485b815c38966d72edf9 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:34.047706 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcc02b-29b0-4c49-8b5f-9ba844e93fae None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:34.048481 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 173/352] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:34.049436 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af4225-5d81-40ed-b476-b69cf9a477a9 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:34.163478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcc02b-29b0-4c49-8b5f-9ba844e93fae None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:34.166537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af4225-5d81-40ed-b476-b69cf9a477a9 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:34.209389 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc403b78-b371-4cf7-a46a-242f3d0665fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:34.209955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc403b78-b371-4cf7-a46a-242f3d0665fb None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:34.210241 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc403b78-b371-4cf7-a46a-242f3d0665fb None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:34.210353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ecff5f-41c3-4923-9dd3-f42bfe4ade5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:34.210565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc403b78-b371-4cf7-a46a-242f3d0665fb None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:34.211161 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecff5f-41c3-4923-9dd3-f42bfe4ade5a None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:34.211452 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecff5f-41c3-4923-9dd3-f42bfe4ade5a None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:34.211731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecff5f-41c3-4923-9dd3-f42bfe4ade5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:34.212348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc403b78-b371-4cf7-a46a-242f3d0665fb None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:34.214023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc403b78-b371-4cf7-a46a-242f3d0665fb None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:34.215795 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 180/353] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:34 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:34.216746 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc403b78-b371-4cf7-a46a-242f3d0665fb None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:34.228162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ecff5f-41c3-4923-9dd3-f42bfe4ade5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:34.236327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-69ecff5f-41c3-4923-9dd3-f42bfe4ade5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:34.236327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:34.236327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:34.236327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:34.236327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:34.236327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:34.236327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:34.236327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:34.236327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:34.236327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:34.236327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:34.236327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:34.236327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:34.236327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:34.236327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:34.236327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:34.237409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:34.237409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:34.237409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:34.237409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:34.237409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:34.237409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:34.237409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:34.237409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:34.237409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:34.237409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:34.237409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:34.237409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:34.237409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:34.237409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:34.237409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:34.237409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:34.238364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:34.238364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:34.238364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:34.238364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:34.238364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:34.238364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:34.238364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:34.238364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:34.238364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:34.238364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:34.238364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:34.238364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:34.240383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 174/354] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:34.241329 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ecff5f-41c3-4923-9dd3-f42bfe4ade5a None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:34.245994 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:34.257305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:34.365892 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc403b78-b371-4cf7-a46a-242f3d0665fb None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:34.380201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ecff5f-41c3-4923-9dd3-f42bfe4ade5a None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:34.417117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d626c9b-82fd-4852-b04e-b206f3ba055f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:34.418193 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d626c9b-82fd-4852-b04e-b206f3ba055f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:34.418504 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d626c9b-82fd-4852-b04e-b206f3ba055f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:34.418785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d626c9b-82fd-4852-b04e-b206f3ba055f None admin] PATH_INFO: `/v3/users/7e041055579f40199f9dc3ebc6c87eb2` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:34.419999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d626c9b-82fd-4852-b04e-b206f3ba055f None admin] RBAC: Authorizing `identity:get_user(user_id=7e041055579f40199f9dc3ebc6c87eb2)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:34.421483 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d626c9b-82fd-4852-b04e-b206f3ba055f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:34.423115 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 181/355] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:52:34 2022] GET /identity/v3/users/7e041055579f40199f9dc3ebc6c87eb2 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:34.424306 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72a730-2bff-41e3-9f28-9a1e9c8d5018 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:34.424590 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72a730-2bff-41e3-9f28-9a1e9c8d5018 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:34.428804 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72a730-2bff-41e3-9f28-9a1e9c8d5018 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:34.428804 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 182/356] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:34.428804 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72a730-2bff-41e3-9f28-9a1e9c8d5018 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:34.436909 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e6efe-fdbf-4735-b7cd-6e1a8533f765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:34.437849 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e6efe-fdbf-4735-b7cd-6e1a8533f765 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:34.438296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e6efe-fdbf-4735-b7cd-6e1a8533f765 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:34.438720 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e6efe-fdbf-4735-b7cd-6e1a8533f765 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:34.439708 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e6efe-fdbf-4735-b7cd-6e1a8533f765 None admin] RBAC: Adding query filter params (group.id=7716588d7cfa485b815c38966d72edf9, role.id=160d4162340e472b9712a7f15cb2ecca, scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:34.440176 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e6efe-fdbf-4735-b7cd-6e1a8533f765 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:34.442096 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e6efe-fdbf-4735-b7cd-6e1a8533f765 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:34.450348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 175/357] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Jul 12 21:52:34 2022] GET /identity/v3/role_assignments?group.id=7716588d7cfa485b815c38966d72edf9&scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5&role.id=160d4162340e472b9712a7f15cb2ecca => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:34.451448 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e6efe-fdbf-4735-b7cd-6e1a8533f765 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:34.526679 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:34.553689 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:34.569868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e6efe-fdbf-4735-b7cd-6e1a8533f765 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:34.572142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72a730-2bff-41e3-9f28-9a1e9c8d5018 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:34.621571 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80486042-221c-4f36-aada-3636d6dcdd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:34.623067 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80486042-221c-4f36-aada-3636d6dcdd37 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:34.623067 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80486042-221c-4f36-aada-3636d6dcdd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:34.623266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80486042-221c-4f36-aada-3636d6dcdd37 None admin] PATH_INFO: `/v3/groups/7716588d7cfa485b815c38966d72edf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:34.625549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80486042-221c-4f36-aada-3636d6dcdd37 None admin] RBAC: Authorizing `identity:get_group(group_id=7716588d7cfa485b815c38966d72edf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:34.626220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f255c5f-ccf9-4fbe-987d-3e8637e72677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:34.626918 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f255c5f-ccf9-4fbe-987d-3e8637e72677 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:34.627251 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f255c5f-ccf9-4fbe-987d-3e8637e72677 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:34.627533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f255c5f-ccf9-4fbe-987d-3e8637e72677 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:34.627627 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80486042-221c-4f36-aada-3636d6dcdd37 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:34.628328 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f255c5f-ccf9-4fbe-987d-3e8637e72677 None admin] RBAC: Adding query filter params (name=member) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:34.628765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f255c5f-ccf9-4fbe-987d-3e8637e72677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:34.629947 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 183/358] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Jul 12 21:52:34 2022] GET /identity/v3/groups/7716588d7cfa485b815c38966d72edf9 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:34.630166 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f255c5f-ccf9-4fbe-987d-3e8637e72677 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:34.631440 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d090b0-ffa1-4c8a-bc29-d664d664b9f5 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:34.631898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d090b0-ffa1-4c8a-bc29-d664d664b9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:34.632179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d090b0-ffa1-4c8a-bc29-d664d664b9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:34.641700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 176/359] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:34.642981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f255c5f-ccf9-4fbe-987d-3e8637e72677 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:34.655159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-97d090b0-ffa1-4c8a-bc29-d664d664b9f5 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:34.656390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-97d090b0-ffa1-4c8a-bc29-d664d664b9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:34.752779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 184/360] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:34.754147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84e1ac-3022-4fd2-91f1-b37f9da8c576 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:34.754436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84e1ac-3022-4fd2-91f1-b37f9da8c576 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:34.754713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84e1ac-3022-4fd2-91f1-b37f9da8c576 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:34.756150 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 185/361] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:34.760879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84e1ac-3022-4fd2-91f1-b37f9da8c576 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:34.786755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f255c5f-ccf9-4fbe-987d-3e8637e72677 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:34.819710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:34.833352 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aa7d9-1fc3-4eca-b63f-ec559c70f0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:34.834030 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aa7d9-1fc3-4eca-b63f-ec559c70f0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:34.834315 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aa7d9-1fc3-4eca-b63f-ec559c70f0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:34.834587 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aa7d9-1fc3-4eca-b63f-ec559c70f0a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:34.836272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46aa7d9-1fc3-4eca-b63f-ec559c70f0a4 None admin] RBAC: Adding query filter params (role.id=d43f8f61eb5f4489acfd5fba99b85919, user.id=7e041055579f40199f9dc3ebc6c87eb2, scope.system=all) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:34.836272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46aa7d9-1fc3-4eca-b63f-ec559c70f0a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:34.837598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46aa7d9-1fc3-4eca-b63f-ec559c70f0a4 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:34.854395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 177/362] 10.222.0.44 () {56 vars in 1308 bytes} [Tue Jul 12 21:52:34 2022] GET /identity/v3/role_assignments?user.id=7e041055579f40199f9dc3ebc6c87eb2&scope.system=all&role.id=d43f8f61eb5f4489acfd5fba99b85919 => generated 510 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:34.859754 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aa7d9-1fc3-4eca-b63f-ec559c70f0a4 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:34.891627 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:34.912337 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84e1ac-3022-4fd2-91f1-b37f9da8c576 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:34.959020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dba1ef-ae9e-43f1-aad1-3fde1bbdec4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:34.959020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dba1ef-ae9e-43f1-aad1-3fde1bbdec4f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:34.959020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dba1ef-ae9e-43f1-aad1-3fde1bbdec4f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:34.959020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dba1ef-ae9e-43f1-aad1-3fde1bbdec4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:34.959414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dba1ef-ae9e-43f1-aad1-3fde1bbdec4f None admin] RBAC: Adding query filter params (group.id=7716588d7cfa485b815c38966d72edf9, role.id=b5ee21738ecb4079977d078d50ddbcf9, scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:34.959596 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dba1ef-ae9e-43f1-aad1-3fde1bbdec4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:34.962448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dba1ef-ae9e-43f1-aad1-3fde1bbdec4f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:34.968809 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 186/363] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Jul 12 21:52:34 2022] GET /identity/v3/role_assignments?group.id=7716588d7cfa485b815c38966d72edf9&scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5&role.id=b5ee21738ecb4079977d078d50ddbcf9 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:34.970239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa34bab-71ec-4eb8-baf5-c45e5fcaae07 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:34.970781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa34bab-71ec-4eb8-baf5-c45e5fcaae07 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:34.971158 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa34bab-71ec-4eb8-baf5-c45e5fcaae07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:34.991267 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-6fa34bab-71ec-4eb8-baf5-c45e5fcaae07 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:34.992656 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-6fa34bab-71ec-4eb8-baf5-c45e5fcaae07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:35.011194 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aa7d9-1fc3-4eca-b63f-ec559c70f0a4 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:35.077332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 187/364] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:35.078379 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa34bab-71ec-4eb8-baf5-c45e5fcaae07 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:35.081561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b96ba-27a9-45c0-b93e-11b9c8d45896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:35.082613 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b96ba-27a9-45c0-b93e-11b9c8d45896 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:35.083143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b96ba-27a9-45c0-b93e-11b9c8d45896 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:35.083660 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b96ba-27a9-45c0-b93e-11b9c8d45896 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:35.183939 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:35.201677 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b96ba-27a9-45c0-b93e-11b9c8d45896 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:35.203728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b96ba-27a9-45c0-b93e-11b9c8d45896 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:35.213339 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:35.228503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa34bab-71ec-4eb8-baf5-c45e5fcaae07 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:35.265664 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d8317-0111-4004-9784-eaadb151d74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:35.266603 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d8317-0111-4004-9784-eaadb151d74e None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:35.266886 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d8317-0111-4004-9784-eaadb151d74e None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:35.267274 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d8317-0111-4004-9784-eaadb151d74e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:35.279406 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d8317-0111-4004-9784-eaadb151d74e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:35.288049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-4c3d8317-0111-4004-9784-eaadb151d74e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:35.288049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:35.288049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:35.288049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:35.288049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:35.288049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:35.288049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:35.288049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:35.288049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:35.288049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:35.288049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:35.288049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:35.288049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:35.288049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:35.288049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:35.288049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:35.288987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:35.288987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:35.288987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:35.288987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:35.288987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:35.288987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:35.288987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:35.288987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:35.288987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:35.288987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:35.288987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:35.288987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:35.288987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:35.288987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:35.288987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:35.288987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:35.289779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:35.289779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:35.289779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:35.289779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:35.289779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:35.289779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:35.289779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:35.289779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:35.289779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:35.289779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:35.289779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:35.289779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:35.289779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 188/365] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:52:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:35.290840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d8317-0111-4004-9784-eaadb151d74e None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:35.341106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 178/366] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:35.341106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b96ba-27a9-45c0-b93e-11b9c8d45896 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:35.433108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d8317-0111-4004-9784-eaadb151d74e None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:35.466305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:35.480663 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b96ba-27a9-45c0-b93e-11b9c8d45896 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:35.486751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb9455-49c5-4642-baad-788bb8b7d252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:35.487829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb9455-49c5-4642-baad-788bb8b7d252 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:35.488313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb9455-49c5-4642-baad-788bb8b7d252 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:35.489319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb9455-49c5-4642-baad-788bb8b7d252 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:35.507436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb9455-49c5-4642-baad-788bb8b7d252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:35.518930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-e2bb9455-49c5-4642-baad-788bb8b7d252 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:52:35.518930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:35.518930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:35.518930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:35.518930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:35.518930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:35.518930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:35.518930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:35.518930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:35.518930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:35.518930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:35.518930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:35.518930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:35.518930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:35.518930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:35.518930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:35.519847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:35.519847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:35.519847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:35.519847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:35.519847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:35.519847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:35.519847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:35.519847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:35.519847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:35.519847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:35.519847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:35.519847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:35.519847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:35.519847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:35.519847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:35.519847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:35.520647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:35.520647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:35.520647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:35.520647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:35.520647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:35.520647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:35.520647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:35.520647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:35.520647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:35.520647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:35.520647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:52:35.520647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:35.521899 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544eb75-12bb-4255-bd2f-d7d91d2c927c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:35.522495 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544eb75-12bb-4255-bd2f-d7d91d2c927c None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:35.522777 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544eb75-12bb-4255-bd2f-d7d91d2c927c None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:35.523054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544eb75-12bb-4255-bd2f-d7d91d2c927c None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:35.523806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544eb75-12bb-4255-bd2f-d7d91d2c927c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:35.524730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 189/367] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:52:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:35.525055 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544eb75-12bb-4255-bd2f-d7d91d2c927c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:35.526105 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb9455-49c5-4642-baad-788bb8b7d252 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:35.526482 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544eb75-12bb-4255-bd2f-d7d91d2c927c None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:35.531201 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:35.537875 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 179/368] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:52:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:35.538520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544eb75-12bb-4255-bd2f-d7d91d2c927c None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:35.672807 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb9455-49c5-4642-baad-788bb8b7d252 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:35.679319 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544eb75-12bb-4255-bd2f-d7d91d2c927c None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:35.717157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff06e8-0f55-44b9-b4b5-c5588c728564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:35.718093 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff06e8-0f55-44b9-b4b5-c5588c728564 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:35.718377 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff06e8-0f55-44b9-b4b5-c5588c728564 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:35.719212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff06e8-0f55-44b9-b4b5-c5588c728564 None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:35.720464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff06e8-0f55-44b9-b4b5-c5588c728564 None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:35.721712 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8e209-2dc9-4f8b-8828-a40201a5ed1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:35.721963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff06e8-0f55-44b9-b4b5-c5588c728564 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:35.722381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8e209-2dc9-4f8b-8828-a40201a5ed1d None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:35.722663 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8e209-2dc9-4f8b-8828-a40201a5ed1d None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:35.722945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8e209-2dc9-4f8b-8828-a40201a5ed1d None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:35.723693 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8e209-2dc9-4f8b-8828-a40201a5ed1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:35.724019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8e209-2dc9-4f8b-8828-a40201a5ed1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:35.724116 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 190/369] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:35 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:35.724850 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff06e8-0f55-44b9-b4b5-c5588c728564 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:35.725387 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8e209-2dc9-4f8b-8828-a40201a5ed1d None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:35.736939 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 180/370] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:35.738680 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8e209-2dc9-4f8b-8828-a40201a5ed1d None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:35.769205 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:35.830467 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:35.867898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff06e8-0f55-44b9-b4b5-c5588c728564 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:35.874815 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8e209-2dc9-4f8b-8828-a40201a5ed1d None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:35.909509 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c913b-4e3d-44d6-9e72-e65d1d6d0dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:35.910165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c913b-4e3d-44d6-9e72-e65d1d6d0dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:35.910453 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c913b-4e3d-44d6-9e72-e65d1d6d0dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:35.910736 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c913b-4e3d-44d6-9e72-e65d1d6d0dc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:35.918985 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9bd031-8edb-4cf5-ac1c-2d6b804aecfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:35.920296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bd031-8edb-4cf5-ac1c-2d6b804aecfb None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:35.921122 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bd031-8edb-4cf5-ac1c-2d6b804aecfb None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:35.921612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bd031-8edb-4cf5-ac1c-2d6b804aecfb None admin] PATH_INFO: `/v3/groups/7716588d7cfa485b815c38966d72edf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:35.923527 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9bd031-8edb-4cf5-ac1c-2d6b804aecfb None admin] RBAC: Authorizing `identity:get_group(group_id=7716588d7cfa485b815c38966d72edf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:35.925892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9bd031-8edb-4cf5-ac1c-2d6b804aecfb None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:35.932934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 181/371] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Jul 12 21:52:35 2022] GET /identity/v3/groups/7716588d7cfa485b815c38966d72edf9 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:35.934226 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9bd031-8edb-4cf5-ac1c-2d6b804aecfb None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:35.990153 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c913b-4e3d-44d6-9e72-e65d1d6d0dc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:35.991646 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c913b-4e3d-44d6-9e72-e65d1d6d0dc5 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:36.049244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:36.074688 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9bd031-8edb-4cf5-ac1c-2d6b804aecfb None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:36.075282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:36.115310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 191/372] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:36.116183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c913b-4e3d-44d6-9e72-e65d1d6d0dc5 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:36.117954 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9e466-4913-411a-afe5-6bfa75059ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:36.118722 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9e466-4913-411a-afe5-6bfa75059ecc None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:36.119111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9e466-4913-411a-afe5-6bfa75059ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:36.119624 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9e466-4913-411a-afe5-6bfa75059ecc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:36.213934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9e466-4913-411a-afe5-6bfa75059ecc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:36.215485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9e466-4913-411a-afe5-6bfa75059ecc None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:36.233367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c913b-4e3d-44d6-9e72-e65d1d6d0dc5 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:36.275193 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f87efb-00db-486f-a736-839b41bd3dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:36.276063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f87efb-00db-486f-a736-839b41bd3dcd None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:36.276351 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f87efb-00db-486f-a736-839b41bd3dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:36.276632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f87efb-00db-486f-a736-839b41bd3dcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:36.277517 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f87efb-00db-486f-a736-839b41bd3dcd None admin] RBAC: Adding query filter params (group.id=7716588d7cfa485b815c38966d72edf9, role.id=b5ee21738ecb4079977d078d50ddbcf9, scope.project.id=bbe948d5cac04aeb909127cea42f77b4) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:36.277837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f87efb-00db-486f-a736-839b41bd3dcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:36.279333 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f87efb-00db-486f-a736-839b41bd3dcd None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:36.285859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 192/373] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Jul 12 21:52:36 2022] GET /identity/v3/role_assignments?group.id=7716588d7cfa485b815c38966d72edf9&scope.project.id=bbe948d5cac04aeb909127cea42f77b4&role.id=b5ee21738ecb4079977d078d50ddbcf9 => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:36.286678 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f87efb-00db-486f-a736-839b41bd3dcd None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:36.332004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:36.336013 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:36.366917 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 182/374] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:36.374624 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9e466-4913-411a-afe5-6bfa75059ecc None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:36.423651 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f87efb-00db-486f-a736-839b41bd3dcd None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:36.466206 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f38148-4854-4f35-80da-67be167f2a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:36.467082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f38148-4854-4f35-80da-67be167f2a3a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:36.467364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f38148-4854-4f35-80da-67be167f2a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:36.467641 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f38148-4854-4f35-80da-67be167f2a3a None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:36.469228 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f38148-4854-4f35-80da-67be167f2a3a None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:36.470706 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f38148-4854-4f35-80da-67be167f2a3a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:36.472854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 193/375] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:36 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:36.479155 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f38148-4854-4f35-80da-67be167f2a3a None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:36.569028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9e466-4913-411a-afe5-6bfa75059ecc None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:36.616251 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f38148-4854-4f35-80da-67be167f2a3a None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:36.619271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:36.633289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9948a93-8687-4487-bd7d-5ab6f1030fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:36.634892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9948a93-8687-4487-bd7d-5ab6f1030fbd None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:36.635225 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9948a93-8687-4487-bd7d-5ab6f1030fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:36.635511 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9948a93-8687-4487-bd7d-5ab6f1030fbd None admin] PATH_INFO: `/v3/projects/3e825823085549c0909b324f2e7acb2a` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:36.641650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9948a93-8687-4487-bd7d-5ab6f1030fbd None admin] RBAC: Authorizing `identity:get_project(project_id=3e825823085549c0909b324f2e7acb2a)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:36.643178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9948a93-8687-4487-bd7d-5ab6f1030fbd None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:36.645421 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 183/376] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:36 2022] GET /identity/v3/projects/3e825823085549c0909b324f2e7acb2a => generated 344 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:36.651126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9948a93-8687-4487-bd7d-5ab6f1030fbd None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:36.673003 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40210e1a-4473-4515-ae05-541c0f0d222c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:36.673966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40210e1a-4473-4515-ae05-541c0f0d222c None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:36.674278 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40210e1a-4473-4515-ae05-541c0f0d222c None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:36.674549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40210e1a-4473-4515-ae05-541c0f0d222c None admin] PATH_INFO: `/v3/groups/7716588d7cfa485b815c38966d72edf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:36.676028 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40210e1a-4473-4515-ae05-541c0f0d222c None admin] RBAC: Authorizing `identity:get_group(group_id=7716588d7cfa485b815c38966d72edf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:36.677733 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40210e1a-4473-4515-ae05-541c0f0d222c None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:36.679864 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 194/377] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Jul 12 21:52:36 2022] GET /identity/v3/groups/7716588d7cfa485b815c38966d72edf9 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:36.693356 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40210e1a-4473-4515-ae05-541c0f0d222c None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:36.736446 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:36.818226 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9948a93-8687-4487-bd7d-5ab6f1030fbd None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:36.877496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2fea2-cd8f-4ab8-a96d-5464830bf217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:36.878318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2fea2-cd8f-4ab8-a96d-5464830bf217 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:36.885077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2fea2-cd8f-4ab8-a96d-5464830bf217 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:36.885077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2fea2-cd8f-4ab8-a96d-5464830bf217 None admin] PATH_INFO: `/v3/groups/2551a4fa92bc46d4b02d5b213b75e48e` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:36.889582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2fea2-cd8f-4ab8-a96d-5464830bf217 None admin] RBAC: Authorizing `identity:get_group(group_id=2551a4fa92bc46d4b02d5b213b75e48e)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:36.891098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2fea2-cd8f-4ab8-a96d-5464830bf217 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:36.894069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 184/378] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Jul 12 21:52:36 2022] GET /identity/v3/groups/2551a4fa92bc46d4b02d5b213b75e48e => generated 229 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:36.904008 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2fea2-cd8f-4ab8-a96d-5464830bf217 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:36.912446 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40210e1a-4473-4515-ae05-541c0f0d222c None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:36.959668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b40915-bda0-4d14-947e-26d55d263f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:36.960335 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b40915-bda0-4d14-947e-26d55d263f6b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:36.960618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b40915-bda0-4d14-947e-26d55d263f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:36.960935 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b40915-bda0-4d14-947e-26d55d263f6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:36.962367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b40915-bda0-4d14-947e-26d55d263f6b None admin] RBAC: Adding query filter params (group.id=7716588d7cfa485b815c38966d72edf9, role.id=160d4162340e472b9712a7f15cb2ecca, scope.project.id=bbe948d5cac04aeb909127cea42f77b4) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:36.962836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b40915-bda0-4d14-947e-26d55d263f6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:36.964606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b40915-bda0-4d14-947e-26d55d263f6b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:36.973076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 195/379] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Jul 12 21:52:36 2022] GET /identity/v3/role_assignments?group.id=7716588d7cfa485b815c38966d72edf9&scope.project.id=bbe948d5cac04aeb909127cea42f77b4&role.id=160d4162340e472b9712a7f15cb2ecca => generated 260 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:37.067948 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2fea2-cd8f-4ab8-a96d-5464830bf217 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:37.088053 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:37.115936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26d495-0d72-4f23-a63d-1db52ea4e18d None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:37.116738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26d495-0d72-4f23-a63d-1db52ea4e18d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:37.117405 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26d495-0d72-4f23-a63d-1db52ea4e18d None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:37.122645 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 196/380] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:37.123203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5e187-4cdd-47de-ad1d-18284ebe2b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:37.123903 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5e187-4cdd-47de-ad1d-18284ebe2b49 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:37.124190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5e187-4cdd-47de-ad1d-18284ebe2b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:37.124472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5e187-4cdd-47de-ad1d-18284ebe2b49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:37.125396 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5e187-4cdd-47de-ad1d-18284ebe2b49 None admin] RBAC: Adding query filter params (group.id=2551a4fa92bc46d4b02d5b213b75e48e, role.id=053d1730ed764973abb1490894e07681, scope.project.id=3e825823085549c0909b324f2e7acb2a) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:37.125716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5e187-4cdd-47de-ad1d-18284ebe2b49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:37.127198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5e187-4cdd-47de-ad1d-18284ebe2b49 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:37.136093 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fefaa-677c-44bd-877c-e4d0a9b2bac2 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:37.136484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fefaa-677c-44bd-877c-e4d0a9b2bac2 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:37.136801 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fefaa-677c-44bd-877c-e4d0a9b2bac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:37.144258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 185/381] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Jul 12 21:52:36 2022] GET /identity/v3/role_assignments?group.id=2551a4fa92bc46d4b02d5b213b75e48e&scope.project.id=3e825823085549c0909b324f2e7acb2a&role.id=053d1730ed764973abb1490894e07681 => generated 260 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:37.162087 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-438fefaa-677c-44bd-877c-e4d0a9b2bac2 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:37.163328 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-438fefaa-677c-44bd-877c-e4d0a9b2bac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:37.213169 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:37.269115 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 197/382] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:37.292917 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5e187-4cdd-47de-ad1d-18284ebe2b49 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:37.446479 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5e187-4cdd-47de-ad1d-18284ebe2b49 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:37.484162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d6051-75bb-445e-b7c8-c06c61af3183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:37.485073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d6051-75bb-445e-b7c8-c06c61af3183 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:37.485351 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d6051-75bb-445e-b7c8-c06c61af3183 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:37.485671 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d6051-75bb-445e-b7c8-c06c61af3183 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:37.504690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1d6051-75bb-445e-b7c8-c06c61af3183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:37.513827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-0b1d6051-75bb-445e-b7c8-c06c61af3183 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:37.513827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:37.513827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:37.513827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:37.513827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:37.513827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:37.513827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:37.513827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:37.513827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:37.513827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:37.513827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:37.513827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:37.513827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:37.513827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:37.513827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:37.513827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:37.516605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:37.516605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:37.516605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:37.516605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:37.516605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:37.516605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:37.516605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:37.516605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:37.516605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:37.516605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:37.516605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:37.516605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:37.516605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:37.516605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:37.516605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:37.516605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:37.518646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:37.518646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:37.518646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:37.518646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:37.518646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:37.518646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:37.518646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:37.518646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:37.518646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:37.518646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:37.518646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:37.518646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:37.518646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 186/383] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:52:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:37.524753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fefaa-677c-44bd-877c-e4d0a9b2bac2 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:37.638218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fefaa-677c-44bd-877c-e4d0a9b2bac2 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:37.674772 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18747e-fffe-40f6-9da5-39cd305d321f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:37.675444 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18747e-fffe-40f6-9da5-39cd305d321f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:37.675724 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18747e-fffe-40f6-9da5-39cd305d321f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:37.675996 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18747e-fffe-40f6-9da5-39cd305d321f None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:37.676782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18747e-fffe-40f6-9da5-39cd305d321f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:37.677228 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18747e-fffe-40f6-9da5-39cd305d321f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:37.678614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18747e-fffe-40f6-9da5-39cd305d321f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:37.688028 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 198/384] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:52:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:37.693711 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d6051-75bb-445e-b7c8-c06c61af3183 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:37.708367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23485e61-cd0a-4a97-974f-ff061027c248 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:37.708677 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23485e61-cd0a-4a97-974f-ff061027c248 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:37.708882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23485e61-cd0a-4a97-974f-ff061027c248 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:37.712456 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 199/385] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:37.718162 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d478b-4f5c-480b-8bdf-290ea278b88e None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:37.718738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d478b-4f5c-480b-8bdf-290ea278b88e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:37.719224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d478b-4f5c-480b-8bdf-290ea278b88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:37.741085 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:37.750064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-4f0d478b-4f5c-480b-8bdf-290ea278b88e None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:37.751929 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-4f0d478b-4f5c-480b-8bdf-290ea278b88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:37.795107 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:37.826223 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d6051-75bb-445e-b7c8-c06c61af3183 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:37.868829 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9828e-54f1-486d-8ba3-b56e1dbbef1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:37.870248 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9828e-54f1-486d-8ba3-b56e1dbbef1f None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:37.870671 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9828e-54f1-486d-8ba3-b56e1dbbef1f None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:37.870965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9828e-54f1-486d-8ba3-b56e1dbbef1f None admin] PATH_INFO: `/v3/groups/7716588d7cfa485b815c38966d72edf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:37.872357 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9828e-54f1-486d-8ba3-b56e1dbbef1f None admin] RBAC: Authorizing `identity:get_group(group_id=7716588d7cfa485b815c38966d72edf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:37.874199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9828e-54f1-486d-8ba3-b56e1dbbef1f None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:37.875914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 187/386] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Jul 12 21:52:37 2022] GET /identity/v3/groups/7716588d7cfa485b815c38966d72edf9 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:37.881978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9828e-54f1-486d-8ba3-b56e1dbbef1f None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:37.943214 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 200/387] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:52:37.985785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d478b-4f5c-480b-8bdf-290ea278b88e None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:37.996262 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9828e-54f1-486d-8ba3-b56e1dbbef1f None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:37.999744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:38.035518 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a386f-7a77-462b-87c6-8eedc944e63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:38.036397 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a386f-7a77-462b-87c6-8eedc944e63d None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:38.036682 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a386f-7a77-462b-87c6-8eedc944e63d None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:38.036990 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a386f-7a77-462b-87c6-8eedc944e63d None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:38.038252 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a386f-7a77-462b-87c6-8eedc944e63d None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:38.040435 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a386f-7a77-462b-87c6-8eedc944e63d None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:38.043372 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 188/388] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:37 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:38.049407 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a386f-7a77-462b-87c6-8eedc944e63d None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:38.166748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a386f-7a77-462b-87c6-8eedc944e63d None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:38.209659 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937152ac-8e3b-41d0-9f05-eb36c966743e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:38.211010 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937152ac-8e3b-41d0-9f05-eb36c966743e None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:38.211441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937152ac-8e3b-41d0-9f05-eb36c966743e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:38.212555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937152ac-8e3b-41d0-9f05-eb36c966743e None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5/groups/7716588d7cfa485b815c38966d72edf9/roles/160d4162340e472b9712a7f15cb2ecca` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:38.216031 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937152ac-8e3b-41d0-9f05-eb36c966743e None admin] RBAC: Authorizing `identity:create_grant(project_id=73157d93d8f74bfba4cf91b3cd885fe5, group_id=7716588d7cfa485b815c38966d72edf9, role_id=160d4162340e472b9712a7f15cb2ecca)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:38.217686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937152ac-8e3b-41d0-9f05-eb36c966743e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:38.236119 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 189/389] 10.222.0.44 () {58 vars in 1442 bytes} [Tue Jul 12 21:52:38 2022] PUT /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5/groups/7716588d7cfa485b815c38966d72edf9/roles/160d4162340e472b9712a7f15cb2ecca => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 21:52:38.314084 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d478b-4f5c-480b-8bdf-290ea278b88e None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:38.373362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5288ec-ef94-4615-abd4-d7b35bf62251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:38.374178 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5288ec-ef94-4615-abd4-d7b35bf62251 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:38.374458 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5288ec-ef94-4615-abd4-d7b35bf62251 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:38.374734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5288ec-ef94-4615-abd4-d7b35bf62251 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:38.386647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5288ec-ef94-4615-abd4-d7b35bf62251 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:38.394946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-ad5288ec-ef94-4615-abd4-d7b35bf62251 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:38.394946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:38.394946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:38.394946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:38.394946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:38.394946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:38.394946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:38.394946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:38.394946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:38.394946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:38.394946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:38.394946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:38.394946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:38.394946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:38.394946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:38.394946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:38.395876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:38.395876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:38.395876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:38.395876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:38.395876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:38.395876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:38.395876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:38.395876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:38.395876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:38.395876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:38.395876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:38.395876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:38.395876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:38.395876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:38.395876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:38.395876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:38.396660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:38.396660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:38.396660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:38.396660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:38.396660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:38.396660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:38.396660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:38.396660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:38.396660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:38.396660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:38.396660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:38.396660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:38.396660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 201/390] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:38.403293 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937152ac-8e3b-41d0-9f05-eb36c966743e None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:38.451484 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:38.536299 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937152ac-8e3b-41d0-9f05-eb36c966743e None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:38.575371 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db72f8-186d-4b4a-8fb0-807b2106ab29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:38.576060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db72f8-186d-4b4a-8fb0-807b2106ab29 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:38.576345 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db72f8-186d-4b4a-8fb0-807b2106ab29 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:38.576620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db72f8-186d-4b4a-8fb0-807b2106ab29 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:38.577418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db72f8-186d-4b4a-8fb0-807b2106ab29 None admin] RBAC: Adding query filter params (name=member) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:38.577722 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db72f8-186d-4b4a-8fb0-807b2106ab29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:38.579114 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db72f8-186d-4b4a-8fb0-807b2106ab29 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:38.589594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 190/391] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:38.595164 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5288ec-ef94-4615-abd4-d7b35bf62251 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:38.635310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:38.769602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5288ec-ef94-4615-abd4-d7b35bf62251 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:38.808999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab96aa-cd26-4b22-9a30-6ac6784c8ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:38.809835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab96aa-cd26-4b22-9a30-6ac6784c8ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:38.810116 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab96aa-cd26-4b22-9a30-6ac6784c8ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:38.810438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab96aa-cd26-4b22-9a30-6ac6784c8ca7 None admin] PATH_INFO: `/v3/groups/7716588d7cfa485b815c38966d72edf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:38.811628 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab96aa-cd26-4b22-9a30-6ac6784c8ca7 None admin] RBAC: Authorizing `identity:get_group(group_id=7716588d7cfa485b815c38966d72edf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:38.813103 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab96aa-cd26-4b22-9a30-6ac6784c8ca7 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:38.814719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 202/392] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Jul 12 21:52:38 2022] GET /identity/v3/groups/7716588d7cfa485b815c38966d72edf9 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:38.820740 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db72f8-186d-4b4a-8fb0-807b2106ab29 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:38.928168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:38.935584 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db72f8-186d-4b4a-8fb0-807b2106ab29 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:38.991288 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1a15c-4142-48e1-ad51-616b84159196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:38.992165 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1a15c-4142-48e1-ad51-616b84159196 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:38.992444 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1a15c-4142-48e1-ad51-616b84159196 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:38.992739 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1a15c-4142-48e1-ad51-616b84159196 None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:38.997477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1a15c-4142-48e1-ad51-616b84159196 None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:38.998928 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1a15c-4142-48e1-ad51-616b84159196 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:39.001246 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 191/393] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:38 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:39.007051 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab96aa-cd26-4b22-9a30-6ac6784c8ca7 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:39.125084 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:39.144926 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab96aa-cd26-4b22-9a30-6ac6784c8ca7 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:39.185523 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bef95-2799-47f9-b93c-c0f0c5c740ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:39.186416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bef95-2799-47f9-b93c-c0f0c5c740ba None admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:39.186698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bef95-2799-47f9-b93c-c0f0c5c740ba None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:39.186973 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bef95-2799-47f9-b93c-c0f0c5c740ba None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5/groups/7716588d7cfa485b815c38966d72edf9/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:39.189023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90bef95-2799-47f9-b93c-c0f0c5c740ba None admin] RBAC: Authorizing `identity:create_grant(project_id=73157d93d8f74bfba4cf91b3cd885fe5, group_id=7716588d7cfa485b815c38966d72edf9, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:39.190518 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90bef95-2799-47f9-b93c-c0f0c5c740ba None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:39.207052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 203/394] 10.222.0.44 () {58 vars in 1442 bytes} [Tue Jul 12 21:52:39 2022] PUT /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5/groups/7716588d7cfa485b815c38966d72edf9/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:52:39.297066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d68ce8-21dc-46ad-be2b-955a69aaa148 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:39.297752 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d68ce8-21dc-46ad-be2b-955a69aaa148 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:39.298234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d68ce8-21dc-46ad-be2b-955a69aaa148 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:39.300272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 192/395] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:39.307853 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b741a8d-9ff6-45c5-aa70-c3df4cef1c45 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:39.308518 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b741a8d-9ff6-45c5-aa70-c3df4cef1c45 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:39.309115 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b741a8d-9ff6-45c5-aa70-c3df4cef1c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:39.337266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-6b741a8d-9ff6-45c5-aa70-c3df4cef1c45 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:39.339458 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-6b741a8d-9ff6-45c5-aa70-c3df4cef1c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:39.437634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 204/396] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:39.472469 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d68ce8-21dc-46ad-be2b-955a69aaa148 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:39.597515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d68ce8-21dc-46ad-be2b-955a69aaa148 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:39.642578 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e399fe-af55-4bf6-95a8-b5e370183004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:39.643765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e399fe-af55-4bf6-95a8-b5e370183004 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:39.644307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e399fe-af55-4bf6-95a8-b5e370183004 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:39.644812 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e399fe-af55-4bf6-95a8-b5e370183004 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:39.659371 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e399fe-af55-4bf6-95a8-b5e370183004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:39.667716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-73e399fe-af55-4bf6-95a8-b5e370183004 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:39.667716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:39.667716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:39.667716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:39.667716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:39.667716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:39.667716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:39.667716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:39.667716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:39.667716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:39.667716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:39.667716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:39.667716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:39.667716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:39.667716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:39.667716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:39.668646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:39.668646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:39.668646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:39.668646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:39.668646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:39.668646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:39.668646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:39.668646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:39.668646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:39.668646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:39.668646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:39.668646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:39.668646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:39.668646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:39.668646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:39.668646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:39.669468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:39.669468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:39.669468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:39.669468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:39.669468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:39.669468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:39.669468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:39.669468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:39.669468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:39.669468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:39.669468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:39.669468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:39.671975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 193/397] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:39.678550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b741a8d-9ff6-45c5-aa70-c3df4cef1c45 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:39.795629 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:39.830833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b741a8d-9ff6-45c5-aa70-c3df4cef1c45 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:39.874371 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ba143-1018-4f93-a452-d05af5b66655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:39.875055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ba143-1018-4f93-a452-d05af5b66655 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:39.875337 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ba143-1018-4f93-a452-d05af5b66655 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:39.875613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ba143-1018-4f93-a452-d05af5b66655 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:39.876376 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ba143-1018-4f93-a452-d05af5b66655 None admin] RBAC: Adding query filter params (name=member) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:39.876685 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ba143-1018-4f93-a452-d05af5b66655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:39.878098 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ba143-1018-4f93-a452-d05af5b66655 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:39.893141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 205/398] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:39.899152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e399fe-af55-4bf6-95a8-b5e370183004 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:39.917946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6f262-b35c-44c1-ada5-01c74e2419b2 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:39.918249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6f262-b35c-44c1-ada5-01c74e2419b2 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:39.918511 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6f262-b35c-44c1-ada5-01c74e2419b2 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:39.920147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 206/399] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:52:39.925111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38485482-3077-4cc5-bc39-647fee0d72db None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:39.925402 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38485482-3077-4cc5-bc39-647fee0d72db None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:39.925685 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38485482-3077-4cc5-bc39-647fee0d72db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:39.950433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-38485482-3077-4cc5-bc39-647fee0d72db None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:39.951636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-38485482-3077-4cc5-bc39-647fee0d72db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:39.953063 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:40.059141 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e399fe-af55-4bf6-95a8-b5e370183004 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:40.060319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 207/400] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:52:40.062697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450421a3-08d5-4d6a-a3a6-ebf89341fd5c None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:40.062986 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450421a3-08d5-4d6a-a3a6-ebf89341fd5c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:40.063266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450421a3-08d5-4d6a-a3a6-ebf89341fd5c None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:40.064549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 208/401] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:40.070693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c3031-a9cc-43c2-afee-a5987b7e925a None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:40.071102 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c3031-a9cc-43c2-afee-a5987b7e925a None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:40.071561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c3031-a9cc-43c2-afee-a5987b7e925a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:40.093020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-352c3031-a9cc-43c2-afee-a5987b7e925a None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:40.094245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-352c3031-a9cc-43c2-afee-a5987b7e925a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:40.118851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4620b-0480-4b81-8a8b-70a301adb9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:40.119791 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4620b-0480-4b81-8a8b-70a301adb9fb None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:40.120173 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4620b-0480-4b81-8a8b-70a301adb9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:40.120454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4620b-0480-4b81-8a8b-70a301adb9fb None admin] PATH_INFO: `/v3/groups/7716588d7cfa485b815c38966d72edf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:40.121878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4620b-0480-4b81-8a8b-70a301adb9fb None admin] RBAC: Authorizing `identity:get_group(group_id=7716588d7cfa485b815c38966d72edf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:40.123458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4620b-0480-4b81-8a8b-70a301adb9fb None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:40.125539 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 194/402] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Jul 12 21:52:39 2022] GET /identity/v3/groups/7716588d7cfa485b815c38966d72edf9 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:40.126487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4620b-0480-4b81-8a8b-70a301adb9fb None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:40.161327 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:40.176734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 209/403] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:40.176734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c3031-a9cc-43c2-afee-a5987b7e925a None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:40.268471 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4620b-0480-4b81-8a8b-70a301adb9fb None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:40.278391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c3031-a9cc-43c2-afee-a5987b7e925a None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:40.288962 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:40.309675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f623145-cc4e-4b2b-8eb0-7ea3bdf0f4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:40.310695 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f623145-cc4e-4b2b-8eb0-7ea3bdf0f4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:40.311225 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f623145-cc4e-4b2b-8eb0-7ea3bdf0f4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:40.311746 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f623145-cc4e-4b2b-8eb0-7ea3bdf0f4b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:40.314420 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bc4b2-4924-4a4d-96e0-88a5471b902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:40.315500 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bc4b2-4924-4a4d-96e0-88a5471b902a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:40.315957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bc4b2-4924-4a4d-96e0-88a5471b902a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:40.316647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bc4b2-4924-4a4d-96e0-88a5471b902a None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:40.318186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bc4b2-4924-4a4d-96e0-88a5471b902a None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:40.319835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bc4b2-4924-4a4d-96e0-88a5471b902a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:40.322272 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 210/404] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:40 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:40.323351 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bc4b2-4924-4a4d-96e0-88a5471b902a None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:40.324389 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f623145-cc4e-4b2b-8eb0-7ea3bdf0f4b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:40.332304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-8f623145-cc4e-4b2b-8eb0-7ea3bdf0f4b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:52:40.332304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:40.332304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:40.332304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:40.332304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:40.332304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:40.332304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:40.332304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:40.332304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:40.332304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:40.332304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:40.332304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:40.332304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:40.332304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:40.332304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:40.332304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:40.333259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:40.333259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:40.333259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:40.333259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:40.333259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:40.333259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:40.333259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:40.333259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:40.333259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:40.333259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:40.333259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:40.333259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:40.333259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:40.333259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:40.333259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:40.333259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:40.334077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:40.334077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:40.334077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:40.334077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:40.334077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:40.334077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:40.334077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:40.334077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:40.334077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:40.334077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:40.334077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:52:40.334077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:40.334077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 195/405] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:52:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:40.335224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f623145-cc4e-4b2b-8eb0-7ea3bdf0f4b7 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:40.392519 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:40.433110 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bc4b2-4924-4a4d-96e0-88a5471b902a None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:40.445836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f623145-cc4e-4b2b-8eb0-7ea3bdf0f4b7 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:40.469674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc15c0c-bab8-4bb9-bcb9-daab3138a845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:40.470800 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc15c0c-bab8-4bb9-bcb9-daab3138a845 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:40.471289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc15c0c-bab8-4bb9-bcb9-daab3138a845 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:40.471777 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc15c0c-bab8-4bb9-bcb9-daab3138a845 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:40.482750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5eb6b-5530-4e46-a259-bad9ca141fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:40.483144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc15c0c-bab8-4bb9-bcb9-daab3138a845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:40.483626 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5eb6b-5530-4e46-a259-bad9ca141fc2 None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:40.483911 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5eb6b-5530-4e46-a259-bad9ca141fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:40.484187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5eb6b-5530-4e46-a259-bad9ca141fc2 None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4/groups/7716588d7cfa485b815c38966d72edf9/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:40.486156 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5eb6b-5530-4e46-a259-bad9ca141fc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=bbe948d5cac04aeb909127cea42f77b4, group_id=7716588d7cfa485b815c38966d72edf9, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:40.487597 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5eb6b-5530-4e46-a259-bad9ca141fc2 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:40.493310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-1cc15c0c-bab8-4bb9-bcb9-daab3138a845 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:40.493310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:40.493310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:40.493310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:40.493310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:40.493310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:40.493310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:40.493310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:40.493310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:40.493310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:40.493310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:40.493310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:40.493310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:40.493310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:40.493310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:40.493310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:40.494382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:40.494382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:40.494382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:40.494382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:40.494382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:40.494382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:40.494382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:40.494382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:40.494382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:40.494382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:40.494382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:40.494382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:40.494382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:40.494382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:40.494382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:40.494382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:40.495170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:40.495170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:40.495170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:40.495170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:40.495170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:40.495170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:40.495170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:40.495170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:40.495170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:40.495170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:40.495170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:40.495170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:40.497747 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 211/406] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:52:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:40.498886 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc15c0c-bab8-4bb9-bcb9-daab3138a845 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:40.510213 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 196/407] 10.222.0.44 () {58 vars in 1442 bytes} [Tue Jul 12 21:52:40 2022] PUT /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4/groups/7716588d7cfa485b815c38966d72edf9/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:52:40.511577 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5eb6b-5530-4e46-a259-bad9ca141fc2 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:40.531007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:40.674515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5eb6b-5530-4e46-a259-bad9ca141fc2 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:40.717542 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a57f60-4310-42ec-9342-43cd7ee24587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:40.718541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a57f60-4310-42ec-9342-43cd7ee24587 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:40.718826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a57f60-4310-42ec-9342-43cd7ee24587 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:40.719138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a57f60-4310-42ec-9342-43cd7ee24587 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:40.720404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a57f60-4310-42ec-9342-43cd7ee24587 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:40.720728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a57f60-4310-42ec-9342-43cd7ee24587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:40.722153 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a57f60-4310-42ec-9342-43cd7ee24587 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:40.735981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 197/408] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:52:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:40.739017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc15c0c-bab8-4bb9-bcb9-daab3138a845 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:40.741507 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a57f60-4310-42ec-9342-43cd7ee24587 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:40.765142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:40.776662 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ad1d4-5d21-4457-b0cd-1a2ec5a4a071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:40.777373 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ad1d4-5d21-4457-b0cd-1a2ec5a4a071 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:40.777667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ad1d4-5d21-4457-b0cd-1a2ec5a4a071 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:40.777954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ad1d4-5d21-4457-b0cd-1a2ec5a4a071 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:40.778715 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ad1d4-5d21-4457-b0cd-1a2ec5a4a071 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:40.779033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ad1d4-5d21-4457-b0cd-1a2ec5a4a071 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:40.780464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ad1d4-5d21-4457-b0cd-1a2ec5a4a071 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:40.791080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 212/409] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:40.796807 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ad1d4-5d21-4457-b0cd-1a2ec5a4a071 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:40.826662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:40.866867 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a57f60-4310-42ec-9342-43cd7ee24587 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:40.913032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7b036-7f0e-49b5-83f5-0fd028effe43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:40.913958 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7b036-7f0e-49b5-83f5-0fd028effe43 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:40.914243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7b036-7f0e-49b5-83f5-0fd028effe43 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:40.914578 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7b036-7f0e-49b5-83f5-0fd028effe43 None admin] PATH_INFO: `/v3/groups/7716588d7cfa485b815c38966d72edf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:40.918542 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7b036-7f0e-49b5-83f5-0fd028effe43 None admin] RBAC: Authorizing `identity:get_group(group_id=7716588d7cfa485b815c38966d72edf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:40.920131 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7b036-7f0e-49b5-83f5-0fd028effe43 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:40.925901 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 198/410] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Jul 12 21:52:40 2022] GET /identity/v3/groups/7716588d7cfa485b815c38966d72edf9 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:40.934679 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7b036-7f0e-49b5-83f5-0fd028effe43 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:40.947312 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ad1d4-5d21-4457-b0cd-1a2ec5a4a071 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:40.997737 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2c58a-b060-49f2-9092-d155323db34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:40.998871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2c58a-b060-49f2-9092-d155323db34b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:40.999163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2c58a-b060-49f2-9092-d155323db34b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:40.999444 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2c58a-b060-49f2-9092-d155323db34b None admin] PATH_INFO: `/v3/groups/2551a4fa92bc46d4b02d5b213b75e48e` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:41.001420 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d2c58a-b060-49f2-9092-d155323db34b None admin] RBAC: Authorizing `identity:get_group(group_id=2551a4fa92bc46d4b02d5b213b75e48e)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:41.002947 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d2c58a-b060-49f2-9092-d155323db34b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:41.005752 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 213/411] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Jul 12 21:52:40 2022] GET /identity/v3/groups/2551a4fa92bc46d4b02d5b213b75e48e => generated 229 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:41.012131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2c58a-b060-49f2-9092-d155323db34b None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:41.111745 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:41.151048 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7b036-7f0e-49b5-83f5-0fd028effe43 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:41.201295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2c58a-b060-49f2-9092-d155323db34b None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:41.217376 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2bcb8-b219-4730-a12e-5d7ec9c3bee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:41.218258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2bcb8-b219-4730-a12e-5d7ec9c3bee6 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:41.218539 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2bcb8-b219-4730-a12e-5d7ec9c3bee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:41.218814 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2bcb8-b219-4730-a12e-5d7ec9c3bee6 None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:41.222030 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2bcb8-b219-4730-a12e-5d7ec9c3bee6 None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:41.225217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2bcb8-b219-4730-a12e-5d7ec9c3bee6 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:41.226973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 199/412] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:40 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:41.242216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2bcb8-b219-4730-a12e-5d7ec9c3bee6 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:41.259974 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:41.273875 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf7af0-cd21-431f-8b14-e9c04d10e350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:41.276032 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf7af0-cd21-431f-8b14-e9c04d10e350 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:41.276332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf7af0-cd21-431f-8b14-e9c04d10e350 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:41.276619 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf7af0-cd21-431f-8b14-e9c04d10e350 None admin] PATH_INFO: `/v3/projects/3e825823085549c0909b324f2e7acb2a` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:41.277982 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf7af0-cd21-431f-8b14-e9c04d10e350 None admin] RBAC: Authorizing `identity:get_project(project_id=3e825823085549c0909b324f2e7acb2a)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:41.279456 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf7af0-cd21-431f-8b14-e9c04d10e350 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:41.286054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 214/413] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:41 2022] GET /identity/v3/projects/3e825823085549c0909b324f2e7acb2a => generated 344 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:41.287677 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1495a0d-9994-4259-bfce-86ac085c03c1 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:41.287972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1495a0d-9994-4259-bfce-86ac085c03c1 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:41.288254 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1495a0d-9994-4259-bfce-86ac085c03c1 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:41.289913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 215/414] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:41.292123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1495a0d-9994-4259-bfce-86ac085c03c1 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:41.367628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2bcb8-b219-4730-a12e-5d7ec9c3bee6 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:41.407565 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f9024-658e-456f-aa85-c1033e6bf63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:41.408454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f9024-658e-456f-aa85-c1033e6bf63d None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:41.408791 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f9024-658e-456f-aa85-c1033e6bf63d None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:41.409049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f9024-658e-456f-aa85-c1033e6bf63d None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4/groups/7716588d7cfa485b815c38966d72edf9/roles/160d4162340e472b9712a7f15cb2ecca` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:41.411293 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f9024-658e-456f-aa85-c1033e6bf63d None admin] RBAC: Authorizing `identity:create_grant(project_id=bbe948d5cac04aeb909127cea42f77b4, group_id=7716588d7cfa485b815c38966d72edf9, role_id=160d4162340e472b9712a7f15cb2ecca)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:41.412772 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f9024-658e-456f-aa85-c1033e6bf63d None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:41.427937 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 200/415] 10.222.0.44 () {58 vars in 1442 bytes} [Tue Jul 12 21:52:41 2022] PUT /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4/groups/7716588d7cfa485b815c38966d72edf9/roles/160d4162340e472b9712a7f15cb2ecca => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:52:41.429003 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1495a0d-9994-4259-bfce-86ac085c03c1 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:41.437133 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802e9de-329f-46cc-99b8-2ec1e298f30f None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:41.437486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802e9de-329f-46cc-99b8-2ec1e298f30f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:41.437774 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802e9de-329f-46cc-99b8-2ec1e298f30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:41.475040 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:41.476613 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-4802e9de-329f-46cc-99b8-2ec1e298f30f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:41.479188 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4802e9de-329f-46cc-99b8-2ec1e298f30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:41.503004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b9f6b-4459-43dc-af1a-71e58d39773d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:41.504195 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9f6b-4459-43dc-af1a-71e58d39773d None admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:41.505263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9f6b-4459-43dc-af1a-71e58d39773d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:41.505695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9f6b-4459-43dc-af1a-71e58d39773d None admin] PATH_INFO: `/v3/projects/3e825823085549c0909b324f2e7acb2a/groups/2551a4fa92bc46d4b02d5b213b75e48e/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:41.511859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b9f6b-4459-43dc-af1a-71e58d39773d None admin] RBAC: Authorizing `identity:create_grant(project_id=3e825823085549c0909b324f2e7acb2a, group_id=2551a4fa92bc46d4b02d5b213b75e48e, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:41.513799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b9f6b-4459-43dc-af1a-71e58d39773d None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:41.541890 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 216/416] 10.222.0.44 () {58 vars in 1442 bytes} [Tue Jul 12 21:52:41 2022] PUT /identity/v3/projects/3e825823085549c0909b324f2e7acb2a/groups/2551a4fa92bc46d4b02d5b213b75e48e/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:52:41.590434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:41.600806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 201/417] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:41.622616 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b9f6b-4459-43dc-af1a-71e58d39773d None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:41.753651 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b9f6b-4459-43dc-af1a-71e58d39773d None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:41.780548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc74f1-4599-4d61-8f3f-e2665216707c None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:41.780902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc74f1-4599-4d61-8f3f-e2665216707c None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:41.781157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc74f1-4599-4d61-8f3f-e2665216707c None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:41.782718 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 202/418] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:41.788237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa0758-32ea-4707-b72d-a2706a90c95c None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:41.788529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa0758-32ea-4707-b72d-a2706a90c95c None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:41.789220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa0758-32ea-4707-b72d-a2706a90c95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:41.807724 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903bee42-d6db-4bed-8437-f73af6fdf110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:41.808945 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bee42-d6db-4bed-8437-f73af6fdf110 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:41.809504 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bee42-d6db-4bed-8437-f73af6fdf110 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:41.810613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bee42-d6db-4bed-8437-f73af6fdf110 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:41.817728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-affa0758-32ea-4707-b72d-a2706a90c95c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:41.818938 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-affa0758-32ea-4707-b72d-a2706a90c95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:41.825502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903bee42-d6db-4bed-8437-f73af6fdf110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:41.834953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-903bee42-d6db-4bed-8437-f73af6fdf110 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:41.834953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:41.834953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:41.834953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:41.834953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:41.834953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:41.834953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:41.834953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:41.834953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:41.834953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:41.834953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:41.834953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:41.834953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:41.834953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:41.834953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:41.834953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:41.835881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:41.835881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:41.835881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:41.835881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:41.835881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:41.835881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:41.835881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:41.835881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:41.835881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:41.835881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:41.835881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:41.835881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:41.835881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:41.835881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:41.835881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:41.835881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:41.837029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:41.837029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:41.837029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:41.837029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:41.837029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:41.837029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:41.837029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:41.837029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:41.837029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:41.837029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:41.837029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:41.837029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:41.844820 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 217/419] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:52:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:41.847287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903bee42-d6db-4bed-8437-f73af6fdf110 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:41.904511 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:41.946964 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 203/420] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:41.989725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903bee42-d6db-4bed-8437-f73af6fdf110 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:41.994472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa0758-32ea-4707-b72d-a2706a90c95c None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:42.039790 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c13650-53ea-4321-a24b-a9c382060718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:42.042702 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c13650-53ea-4321-a24b-a9c382060718 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:42.044631 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c13650-53ea-4321-a24b-a9c382060718 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:42.045154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c13650-53ea-4321-a24b-a9c382060718 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:42.046151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c13650-53ea-4321-a24b-a9c382060718 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:42.047705 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c13650-53ea-4321-a24b-a9c382060718 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:42.049389 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c13650-53ea-4321-a24b-a9c382060718 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:42.068963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 218/421] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:52:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:42.074721 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c13650-53ea-4321-a24b-a9c382060718 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:42.221582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa0758-32ea-4707-b72d-a2706a90c95c None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:42.233692 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c13650-53ea-4321-a24b-a9c382060718 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:42.236343 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:42.271126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:42.281318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c222d57-bda3-4c9b-a513-66efa0592e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:42.282124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c222d57-bda3-4c9b-a513-66efa0592e30 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:42.282407 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c222d57-bda3-4c9b-a513-66efa0592e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:42.282685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c222d57-bda3-4c9b-a513-66efa0592e30 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:42.290468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0ded0-e148-46c9-8024-f806aaa07809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:42.291062 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0ded0-e148-46c9-8024-f806aaa07809 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:42.291350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0ded0-e148-46c9-8024-f806aaa07809 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:42.291617 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0ded0-e148-46c9-8024-f806aaa07809 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:42.293878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c222d57-bda3-4c9b-a513-66efa0592e30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:42.302038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-4c222d57-bda3-4c9b-a513-66efa0592e30 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:42.302038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:42.302038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:42.302038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:42.302038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:42.302038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:42.302038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:42.302038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:42.302038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:42.302038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:42.302038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:42.302038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:42.302038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:42.302038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:42.302038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:42.302038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:42.302934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:42.302934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:42.302934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:42.302934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:42.302934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:42.302934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:42.302934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:42.302934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:42.302934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:42.302934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:42.302934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:42.302934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:42.302934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:42.302934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:42.302934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:42.302934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:42.303947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:42.303947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:42.303947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:42.303947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:42.303947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:42.303947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:42.303947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:42.303947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:42.303947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:42.303947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:42.303947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:42.303947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:42.303947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 204/422] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:42.312369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c222d57-bda3-4c9b-a513-66efa0592e30 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:42.381468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0ded0-e148-46c9-8024-f806aaa07809 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:42.382984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0ded0-e148-46c9-8024-f806aaa07809 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:42.452168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c222d57-bda3-4c9b-a513-66efa0592e30 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:42.489449 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aca247-a65d-435d-95df-eb03bd06062b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:42.490122 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aca247-a65d-435d-95df-eb03bd06062b None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:42.490405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aca247-a65d-435d-95df-eb03bd06062b None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:42.490680 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aca247-a65d-435d-95df-eb03bd06062b None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:42.491428 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aca247-a65d-435d-95df-eb03bd06062b None admin] RBAC: Adding query filter params (name=member) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:42.491730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aca247-a65d-435d-95df-eb03bd06062b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:42.493893 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aca247-a65d-435d-95df-eb03bd06062b None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:42.503143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 205/423] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:42.508558 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aca247-a65d-435d-95df-eb03bd06062b None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:42.563935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:42.573651 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 219/424] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:42.579163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0ded0-e148-46c9-8024-f806aaa07809 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:42.582460 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:42.627222 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aca247-a65d-435d-95df-eb03bd06062b None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:42.662644 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b08dc5-2fe6-4740-be24-4c9cb6adac8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:42.663445 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b08dc5-2fe6-4740-be24-4c9cb6adac8d None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:42.663824 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b08dc5-2fe6-4740-be24-4c9cb6adac8d None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:42.664199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b08dc5-2fe6-4740-be24-4c9cb6adac8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:42.741427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b08dc5-2fe6-4740-be24-4c9cb6adac8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:42.741427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b08dc5-2fe6-4740-be24-4c9cb6adac8d None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:42.801061 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:42.807917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0ded0-e148-46c9-8024-f806aaa07809 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:42.867503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50f14b-8388-405d-ab17-1227e6a9228f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:42.869227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50f14b-8388-405d-ab17-1227e6a9228f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:42.869697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50f14b-8388-405d-ab17-1227e6a9228f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:42.870128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50f14b-8388-405d-ab17-1227e6a9228f None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:42.871949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50f14b-8388-405d-ab17-1227e6a9228f None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:42.873461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50f14b-8388-405d-ab17-1227e6a9228f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:42.875599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 220/425] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:42 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:42.893295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50f14b-8388-405d-ab17-1227e6a9228f None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:42.893876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 206/426] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 21:52:42.899743 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b08dc5-2fe6-4740-be24-4c9cb6adac8d None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:43.009258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b08dc5-2fe6-4740-be24-4c9cb6adac8d None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:43.048878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d3d9c-0f5b-463d-8b0a-adf9dbe94fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:43.050076 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d3d9c-0f5b-463d-8b0a-adf9dbe94fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:43.050485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d3d9c-0f5b-463d-8b0a-adf9dbe94fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:43.050776 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d3d9c-0f5b-463d-8b0a-adf9dbe94fb9 None admin] PATH_INFO: `/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:43.052106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d3d9c-0f5b-463d-8b0a-adf9dbe94fb9 None admin] RBAC: Authorizing `identity:get_project(project_id=73157d93d8f74bfba4cf91b3cd885fe5)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:43.053749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d3d9c-0f5b-463d-8b0a-adf9dbe94fb9 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:43.056420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 207/427] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:42 2022] GET /identity/v3/projects/73157d93d8f74bfba4cf91b3cd885fe5 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:43.061808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d3d9c-0f5b-463d-8b0a-adf9dbe94fb9 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:43.074263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:43.087395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:43.156675 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50f14b-8388-405d-ab17-1227e6a9228f None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:43.185342 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d3d9c-0f5b-463d-8b0a-adf9dbe94fb9 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:43.204008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c92b9f-09d6-4e8c-8bdd-bb5dc1e0724b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:43.205279 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c92b9f-09d6-4e8c-8bdd-bb5dc1e0724b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:43.205564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c92b9f-09d6-4e8c-8bdd-bb5dc1e0724b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:43.205841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c92b9f-09d6-4e8c-8bdd-bb5dc1e0724b None admin] PATH_INFO: `/v3/groups/7716588d7cfa485b815c38966d72edf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:43.207014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c92b9f-09d6-4e8c-8bdd-bb5dc1e0724b None admin] RBAC: Authorizing `identity:get_group(group_id=7716588d7cfa485b815c38966d72edf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:43.208464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c92b9f-09d6-4e8c-8bdd-bb5dc1e0724b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:43.211272 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 221/428] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Jul 12 21:52:42 2022] GET /identity/v3/groups/7716588d7cfa485b815c38966d72edf9 => generated 226 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:43.213185 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66225a-020b-4500-bc65-dd5a036ceb6a None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:43.213475 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66225a-020b-4500-bc65-dd5a036ceb6a None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:43.214180 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66225a-020b-4500-bc65-dd5a036ceb6a None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:43.219664 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 222/429] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:43.219664 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d66225a-020b-4500-bc65-dd5a036ceb6a None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:43.237401 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ea7a1-1528-4230-9930-e60a2d666cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:43.238741 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ea7a1-1528-4230-9930-e60a2d666cea None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:43.239027 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ea7a1-1528-4230-9930-e60a2d666cea None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:43.239303 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ea7a1-1528-4230-9930-e60a2d666cea None admin] PATH_INFO: `/v3/groups/7716588d7cfa485b815c38966d72edf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:43.240504 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ea7a1-1528-4230-9930-e60a2d666cea None admin] RBAC: Authorizing `identity:get_group(group_id=7716588d7cfa485b815c38966d72edf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:43.242537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ea7a1-1528-4230-9930-e60a2d666cea None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:43.245066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 208/430] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Jul 12 21:52:43 2022] GET /identity/v3/groups/7716588d7cfa485b815c38966d72edf9 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:43.249409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c05d1-d6a3-4202-803b-71536ff26339 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:43.249960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c05d1-d6a3-4202-803b-71536ff26339 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:43.250527 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c05d1-d6a3-4202-803b-71536ff26339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:43.271882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-bf6c05d1-d6a3-4202-803b-71536ff26339 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:43.273457 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-bf6c05d1-d6a3-4202-803b-71536ff26339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:43.351489 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 209/431] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:43.357785 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c05d1-d6a3-4202-803b-71536ff26339 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:43.363763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d66225a-020b-4500-bc65-dd5a036ceb6a None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:43.392399 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:43.410977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9939e5f-6f4d-4424-8261-abd858aa8187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:43.411656 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9939e5f-6f4d-4424-8261-abd858aa8187 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:43.412007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9939e5f-6f4d-4424-8261-abd858aa8187 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:43.412355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9939e5f-6f4d-4424-8261-abd858aa8187 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:43.413298 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9939e5f-6f4d-4424-8261-abd858aa8187 None admin] RBAC: Adding query filter params (group.id=7716588d7cfa485b815c38966d72edf9, role.id=160d4162340e472b9712a7f15cb2ecca, scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:43.413675 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9939e5f-6f4d-4424-8261-abd858aa8187 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:43.415734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9939e5f-6f4d-4424-8261-abd858aa8187 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:43.421961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 223/432] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Jul 12 21:52:43 2022] GET /identity/v3/role_assignments?group.id=7716588d7cfa485b815c38966d72edf9&scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5&role.id=160d4162340e472b9712a7f15cb2ecca => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:43.422911 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9939e5f-6f4d-4424-8261-abd858aa8187 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:43.427752 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:43.511534 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c05d1-d6a3-4202-803b-71536ff26339 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:43.557109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc14b9a-7727-455b-a285-906fc4be699e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:43.558091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc14b9a-7727-455b-a285-906fc4be699e None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:43.558553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc14b9a-7727-455b-a285-906fc4be699e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:43.558998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc14b9a-7727-455b-a285-906fc4be699e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:43.560043 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc14b9a-7727-455b-a285-906fc4be699e None admin] RBAC: Adding query filter params (group.id=7716588d7cfa485b815c38966d72edf9, role.id=b5ee21738ecb4079977d078d50ddbcf9, scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:43.560545 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc14b9a-7727-455b-a285-906fc4be699e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:43.563389 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc14b9a-7727-455b-a285-906fc4be699e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:43.572306 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 210/433] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Jul 12 21:52:43 2022] GET /identity/v3/role_assignments?group.id=7716588d7cfa485b815c38966d72edf9&scope.project.id=73157d93d8f74bfba4cf91b3cd885fe5&role.id=b5ee21738ecb4079977d078d50ddbcf9 => generated 611 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:43.635640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9939e5f-6f4d-4424-8261-abd858aa8187 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:43.676846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f0557-d8bd-4207-9255-b5c0e7df3f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:43.677586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f0557-d8bd-4207-9255-b5c0e7df3f61 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:43.677866 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f0557-d8bd-4207-9255-b5c0e7df3f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:43.678145 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f0557-d8bd-4207-9255-b5c0e7df3f61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:43.690454 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f0557-d8bd-4207-9255-b5c0e7df3f61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:43.698734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-277f0557-d8bd-4207-9255-b5c0e7df3f61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:43.698734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:43.698734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:43.698734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:43.698734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:43.698734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:43.698734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:43.698734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:43.698734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:43.698734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:43.698734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:43.698734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:43.698734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:43.698734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:43.698734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:43.698734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:43.699679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:43.699679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:43.699679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:43.699679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:43.699679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:43.699679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:43.699679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:43.699679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:43.699679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:43.699679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:43.699679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:43.699679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:43.699679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:43.699679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:43.699679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:43.699679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:43.700464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:43.700464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:43.700464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:43.700464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:43.700464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:43.700464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:43.700464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:43.700464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:43.700464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:43.700464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:43.700464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 21:52:43.700464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:43.704922 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 224/434] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:43.706330 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc14b9a-7727-455b-a285-906fc4be699e None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:43.815554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:43.837701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc14b9a-7727-455b-a285-906fc4be699e None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:43.876895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027ee3a-f904-470b-b9de-2defb1662a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:43.877856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027ee3a-f904-470b-b9de-2defb1662a61 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:43.878309 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027ee3a-f904-470b-b9de-2defb1662a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:43.878758 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027ee3a-f904-470b-b9de-2defb1662a61 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:43.879692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027ee3a-f904-470b-b9de-2defb1662a61 None admin] RBAC: Adding query filter params (name=member) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:43.880201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027ee3a-f904-470b-b9de-2defb1662a61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:43.881806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027ee3a-f904-470b-b9de-2defb1662a61 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:43.895876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 211/435] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:52:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:43.898757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f0557-d8bd-4207-9255-b5c0e7df3f61 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:43.975383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:44.013870 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b561d33-3742-4fe1-b65d-82c8009f4158 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:44.014410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b561d33-3742-4fe1-b65d-82c8009f4158 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:44.014410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b561d33-3742-4fe1-b65d-82c8009f4158 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:44.017168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 212/436] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:44.022803 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b5c09-3ade-4b37-abe9-9ac4bd8085dc None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:44.023213 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b5c09-3ade-4b37-abe9-9ac4bd8085dc None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:44.023574 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b5c09-3ade-4b37-abe9-9ac4bd8085dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:44.025746 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f0557-d8bd-4207-9255-b5c0e7df3f61 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:44.045334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-850b5c09-3ade-4b37-abe9-9ac4bd8085dc None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:44.046831 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-850b5c09-3ade-4b37-abe9-9ac4bd8085dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:44.063647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24c224-42a3-4a1a-806e-423893c0572f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:44.064312 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c224-42a3-4a1a-806e-423893c0572f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:44.064602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c224-42a3-4a1a-806e-423893c0572f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:44.064910 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c224-42a3-4a1a-806e-423893c0572f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:44.135633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 213/437] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:52:44.137077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf746fc6-2bf2-4148-9fff-ab727e30c268 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:44.137370 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf746fc6-2bf2-4148-9fff-ab727e30c268 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:44.137650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf746fc6-2bf2-4148-9fff-ab727e30c268 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:44.139215 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 214/438] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:44.144414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47e593-3551-4796-8ab1-2927693c7ab5 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:44.144704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47e593-3551-4796-8ab1-2927693c7ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:44.145012 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47e593-3551-4796-8ab1-2927693c7ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:44.159327 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24c224-42a3-4a1a-806e-423893c0572f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:44.161288 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24c224-42a3-4a1a-806e-423893c0572f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:44.162438 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-4b47e593-3551-4796-8ab1-2927693c7ab5 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:44.163631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4b47e593-3551-4796-8ab1-2927693c7ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:44.239163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:44.243493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 215/439] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:52:44.244311 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b47e593-3551-4796-8ab1-2927693c7ab5 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:44.247408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:44.286266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 225/440] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:44.287243 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24c224-42a3-4a1a-806e-423893c0572f None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:44.359272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b47e593-3551-4796-8ab1-2927693c7ab5 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:44.392633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24c224-42a3-4a1a-806e-423893c0572f None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:44.393960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc46e4-aea8-4780-b977-dc0fa84a9e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:44.394762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc46e4-aea8-4780-b977-dc0fa84a9e95 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:44.395066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc46e4-aea8-4780-b977-dc0fa84a9e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:44.395330 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc46e4-aea8-4780-b977-dc0fa84a9e95 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:44.405375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc46e4-aea8-4780-b977-dc0fa84a9e95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:44.413441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-91fc46e4-aea8-4780-b977-dc0fa84a9e95 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:44.413441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:44.413441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:44.413441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:44.413441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:44.413441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:44.413441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:44.413441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:44.413441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:44.413441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:44.413441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:44.413441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:44.413441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:44.413441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:44.413441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:44.413441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:44.414362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:44.414362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:44.414362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:44.414362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:44.414362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:44.414362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:44.414362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:44.414362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:44.414362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:44.414362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:44.414362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:44.414362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:44.414362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:44.414362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:44.414362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:44.414362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:44.415341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:44.415341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:44.415341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:44.415341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:44.415341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:44.415341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:44.415341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:44.415341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:44.415341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:44.415341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:44.415341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:44.415341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:44.415341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 216/441] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:52:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:44.416203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc46e4-aea8-4780-b977-dc0fa84a9e95 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:44.431564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae529f2-61fd-46f7-83cc-684d71075707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:44.432364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae529f2-61fd-46f7-83cc-684d71075707 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:44.432648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae529f2-61fd-46f7-83cc-684d71075707 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:44.432954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae529f2-61fd-46f7-83cc-684d71075707 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:44.443220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae529f2-61fd-46f7-83cc-684d71075707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:44.451196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-2ae529f2-61fd-46f7-83cc-684d71075707 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:52:44.451196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:44.451196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:44.451196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:44.451196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:44.451196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:44.451196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:44.451196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:44.451196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:44.451196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:44.451196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:44.451196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:44.451196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:44.451196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:44.451196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:44.451196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:44.452224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:44.452224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:44.452224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:44.452224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:44.452224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:44.452224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:44.452224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:44.452224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:44.452224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:44.452224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:44.452224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:44.452224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:44.452224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:44.452224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:44.452224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:44.452224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:44.453260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:44.453260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:44.453260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:44.453260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:44.453260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:44.453260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:44.453260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:44.453260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:44.453260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:44.453260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:44.453260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:52:44.453260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:44.453260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 226/442] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:52:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:44.453260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae529f2-61fd-46f7-83cc-684d71075707 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:44.479750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:44.482505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:44.519627 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc46e4-aea8-4780-b977-dc0fa84a9e95 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:44.553131 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b95d97-cff7-4b0b-a5e0-30555cc34f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:44.553983 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b95d97-cff7-4b0b-a5e0-30555cc34f79 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:44.554261 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b95d97-cff7-4b0b-a5e0-30555cc34f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:44.554529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b95d97-cff7-4b0b-a5e0-30555cc34f79 None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:44.556254 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b95d97-cff7-4b0b-a5e0-30555cc34f79 None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:44.557254 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae529f2-61fd-46f7-83cc-684d71075707 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:44.557724 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b95d97-cff7-4b0b-a5e0-30555cc34f79 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:44.563785 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 217/443] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:44 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:44.563785 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b95d97-cff7-4b0b-a5e0-30555cc34f79 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:44.592989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54ce37-cb29-4f39-b222-d4f127af5ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:44.593663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ce37-cb29-4f39-b222-d4f127af5ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:44.593949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ce37-cb29-4f39-b222-d4f127af5ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:44.594231 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ce37-cb29-4f39-b222-d4f127af5ad8 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:44.594981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54ce37-cb29-4f39-b222-d4f127af5ad8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:44.595292 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54ce37-cb29-4f39-b222-d4f127af5ad8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:44.596677 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54ce37-cb29-4f39-b222-d4f127af5ad8 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:44.605202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 227/444] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:52:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:44.606004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54ce37-cb29-4f39-b222-d4f127af5ad8 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:44.665264 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b95d97-cff7-4b0b-a5e0-30555cc34f79 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:44.694998 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:44.700421 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:44.706787 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd1600-79f9-4e8d-86f5-ac3f6f7452ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:44.707498 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd1600-79f9-4e8d-86f5-ac3f6f7452ad None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:44.707857 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd1600-79f9-4e8d-86f5-ac3f6f7452ad None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:44.708065 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd1600-79f9-4e8d-86f5-ac3f6f7452ad None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:44.708899 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd1600-79f9-4e8d-86f5-ac3f6f7452ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:44.709378 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd1600-79f9-4e8d-86f5-ac3f6f7452ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:44.710798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd1600-79f9-4e8d-86f5-ac3f6f7452ad None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:44.713125 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54ce37-cb29-4f39-b222-d4f127af5ad8 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:44.719594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 218/445] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:44.720436 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd1600-79f9-4e8d-86f5-ac3f6f7452ad None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:44.752489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c5ef9-bb71-4916-a4bf-efb1a5311874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:44.753449 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c5ef9-bb71-4916-a4bf-efb1a5311874 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:44.753803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c5ef9-bb71-4916-a4bf-efb1a5311874 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:44.754148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c5ef9-bb71-4916-a4bf-efb1a5311874 None admin] PATH_INFO: `/v3/groups/7716588d7cfa485b815c38966d72edf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:44.755333 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c5ef9-bb71-4916-a4bf-efb1a5311874 None admin] RBAC: Authorizing `identity:get_group(group_id=7716588d7cfa485b815c38966d72edf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:44.756833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c5ef9-bb71-4916-a4bf-efb1a5311874 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:44.758489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 228/446] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Jul 12 21:52:44 2022] GET /identity/v3/groups/7716588d7cfa485b815c38966d72edf9 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:44.759368 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c5ef9-bb71-4916-a4bf-efb1a5311874 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:44.824467 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd1600-79f9-4e8d-86f5-ac3f6f7452ad None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:44.858493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f71d9e-759f-4a15-9a70-9d77987bcd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:44.859138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f71d9e-759f-4a15-9a70-9d77987bcd57 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:44.859423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f71d9e-759f-4a15-9a70-9d77987bcd57 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:44.859538 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c5ef9-bb71-4916-a4bf-efb1a5311874 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:44.859702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f71d9e-759f-4a15-9a70-9d77987bcd57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:44.893200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cd683-8a8d-4415-b559-ac5ca390c7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:44.893849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cd683-8a8d-4415-b559-ac5ca390c7da None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:44.894135 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cd683-8a8d-4415-b559-ac5ca390c7da None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:44.894411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cd683-8a8d-4415-b559-ac5ca390c7da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:44.919582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:44.927805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:44.937681 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f71d9e-759f-4a15-9a70-9d77987bcd57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:44.939155 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f71d9e-759f-4a15-9a70-9d77987bcd57 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:44.965535 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cd683-8a8d-4415-b559-ac5ca390c7da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:44.967802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cd683-8a8d-4415-b559-ac5ca390c7da None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:45.047575 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 219/447] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:45.048426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f71d9e-759f-4a15-9a70-9d77987bcd57 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:45.073310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 229/448] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:45.074020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cd683-8a8d-4415-b559-ac5ca390c7da None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:45.129131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:45.145474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:45.155086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f71d9e-759f-4a15-9a70-9d77987bcd57 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:45.175660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cd683-8a8d-4415-b559-ac5ca390c7da None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:45.192430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bcd54-beb0-4ed4-8181-6485480f1f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:45.193118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bcd54-beb0-4ed4-8181-6485480f1f13 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:45.193403 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bcd54-beb0-4ed4-8181-6485480f1f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:45.193678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bcd54-beb0-4ed4-8181-6485480f1f13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:45.194512 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210bcd54-beb0-4ed4-8181-6485480f1f13 None admin] RBAC: Adding query filter params (group.id=7716588d7cfa485b815c38966d72edf9, role.id=b5ee21738ecb4079977d078d50ddbcf9, scope.project.id=bbe948d5cac04aeb909127cea42f77b4) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:45.194823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210bcd54-beb0-4ed4-8181-6485480f1f13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:45.196252 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210bcd54-beb0-4ed4-8181-6485480f1f13 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:45.202367 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 220/449] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Jul 12 21:52:45 2022] GET /identity/v3/role_assignments?group.id=7716588d7cfa485b815c38966d72edf9&scope.project.id=bbe948d5cac04aeb909127cea42f77b4&role.id=b5ee21738ecb4079977d078d50ddbcf9 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:45.203247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bcd54-beb0-4ed4-8181-6485480f1f13 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:45.211930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215af4e-ceaa-4290-aeb8-57f254c96aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:45.212877 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215af4e-ceaa-4290-aeb8-57f254c96aea None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:45.213164 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215af4e-ceaa-4290-aeb8-57f254c96aea None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:45.213447 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215af4e-ceaa-4290-aeb8-57f254c96aea None admin] PATH_INFO: `/v3/projects/bbe948d5cac04aeb909127cea42f77b4` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:45.214766 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215af4e-ceaa-4290-aeb8-57f254c96aea None admin] RBAC: Authorizing `identity:get_project(project_id=bbe948d5cac04aeb909127cea42f77b4)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:45.216261 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215af4e-ceaa-4290-aeb8-57f254c96aea None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:45.218109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 230/450] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:45 2022] GET /identity/v3/projects/bbe948d5cac04aeb909127cea42f77b4 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:45.223832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215af4e-ceaa-4290-aeb8-57f254c96aea None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:45.336182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bcd54-beb0-4ed4-8181-6485480f1f13 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:45.342946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215af4e-ceaa-4290-aeb8-57f254c96aea None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:45.369884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:45.376323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce560607-332d-4fa3-9ce7-f5698f2b11f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:45.377225 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce560607-332d-4fa3-9ce7-f5698f2b11f5 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:45.377503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce560607-332d-4fa3-9ce7-f5698f2b11f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:45.377778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce560607-332d-4fa3-9ce7-f5698f2b11f5 None admin] PATH_INFO: `/v3/projects/3e825823085549c0909b324f2e7acb2a` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:45.378959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce560607-332d-4fa3-9ce7-f5698f2b11f5 None admin] RBAC: Authorizing `identity:get_project(project_id=3e825823085549c0909b324f2e7acb2a)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:45.381321 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce560607-332d-4fa3-9ce7-f5698f2b11f5 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:45.382968 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 221/451] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:52:45 2022] GET /identity/v3/projects/3e825823085549c0909b324f2e7acb2a => generated 344 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:45.387151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f7a46-a33d-4d2a-b6c9-4739e143a992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:45.388015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f7a46-a33d-4d2a-b6c9-4739e143a992 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:45.388739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f7a46-a33d-4d2a-b6c9-4739e143a992 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:45.388739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f7a46-a33d-4d2a-b6c9-4739e143a992 None admin] PATH_INFO: `/v3/groups/7716588d7cfa485b815c38966d72edf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:45.389032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce560607-332d-4fa3-9ce7-f5698f2b11f5 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:45.389867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f7a46-a33d-4d2a-b6c9-4739e143a992 None admin] RBAC: Authorizing `identity:get_group(group_id=7716588d7cfa485b815c38966d72edf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:45.392047 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f7a46-a33d-4d2a-b6c9-4739e143a992 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:45.393080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 231/452] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Jul 12 21:52:45 2022] GET /identity/v3/groups/7716588d7cfa485b815c38966d72edf9 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:45.398734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f7a46-a33d-4d2a-b6c9-4739e143a992 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:45.422898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:45.518540 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f7a46-a33d-4d2a-b6c9-4739e143a992 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:45.521190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce560607-332d-4fa3-9ce7-f5698f2b11f5 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:45.564549 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cfce4-235e-46a0-8efe-f59fbe7acd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:45.565686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cfce4-235e-46a0-8efe-f59fbe7acd22 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:45.565973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cfce4-235e-46a0-8efe-f59fbe7acd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:45.566254 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cfce4-235e-46a0-8efe-f59fbe7acd22 None admin] PATH_INFO: `/v3/groups/2551a4fa92bc46d4b02d5b213b75e48e` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:45.567921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d2e15-0ed9-4ff9-aa6d-8a840ca6a833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:45.568336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cfce4-235e-46a0-8efe-f59fbe7acd22 None admin] RBAC: Authorizing `identity:get_group(group_id=2551a4fa92bc46d4b02d5b213b75e48e)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:45.568663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d2e15-0ed9-4ff9-aa6d-8a840ca6a833 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:45.569076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d2e15-0ed9-4ff9-aa6d-8a840ca6a833 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:45.569358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d2e15-0ed9-4ff9-aa6d-8a840ca6a833 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:45.569879 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cfce4-235e-46a0-8efe-f59fbe7acd22 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:45.570192 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d2e15-0ed9-4ff9-aa6d-8a840ca6a833 None admin] RBAC: Adding query filter params (group.id=7716588d7cfa485b815c38966d72edf9, role.id=160d4162340e472b9712a7f15cb2ecca, scope.project.id=bbe948d5cac04aeb909127cea42f77b4) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:45.570502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d2e15-0ed9-4ff9-aa6d-8a840ca6a833 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:45.572474 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d2e15-0ed9-4ff9-aa6d-8a840ca6a833 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:45.576817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 222/453] 10.222.0.44 () {56 vars in 1180 bytes} [Tue Jul 12 21:52:45 2022] GET /identity/v3/groups/2551a4fa92bc46d4b02d5b213b75e48e => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:45.579722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 232/454] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Jul 12 21:52:45 2022] GET /identity/v3/role_assignments?group.id=7716588d7cfa485b815c38966d72edf9&scope.project.id=bbe948d5cac04aeb909127cea42f77b4&role.id=160d4162340e472b9712a7f15cb2ecca => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:45.579843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cfce4-235e-46a0-8efe-f59fbe7acd22 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:45.686031 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:45.695597 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cfce4-235e-46a0-8efe-f59fbe7acd22 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:45.732736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c4a96-627b-4436-9454-970e8d6e53aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:45.733419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c4a96-627b-4436-9454-970e8d6e53aa None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:45.733702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c4a96-627b-4436-9454-970e8d6e53aa None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:45.733977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c4a96-627b-4436-9454-970e8d6e53aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:45.735670 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7c4a96-627b-4436-9454-970e8d6e53aa None admin] RBAC: Adding query filter params (group.id=2551a4fa92bc46d4b02d5b213b75e48e, role.id=053d1730ed764973abb1490894e07681, scope.project.id=3e825823085549c0909b324f2e7acb2a) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:45.735670 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7c4a96-627b-4436-9454-970e8d6e53aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:45.736560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7c4a96-627b-4436-9454-970e8d6e53aa None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:45.743879 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 223/455] 10.222.0.44 () {56 vars in 1376 bytes} [Tue Jul 12 21:52:45 2022] GET /identity/v3/role_assignments?group.id=2551a4fa92bc46d4b02d5b213b75e48e&scope.project.id=3e825823085549c0909b324f2e7acb2a&role.id=053d1730ed764973abb1490894e07681 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:48.486793 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d6b39-cc10-4d92-abad-f4cc04633f0b None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:48.487315 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d6b39-cc10-4d92-abad-f4cc04633f0b None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:48.487415 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d6b39-cc10-4d92-abad-f4cc04633f0b None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:48.489009 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 233/456] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:48.494373 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702f872-0f96-45a1-81d2-fc2a5a5df7dd None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:48.496429 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e97bb2-9c23-4e0f-83c3-1a34595e00f8 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:48.496967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e97bb2-9c23-4e0f-83c3-1a34595e00f8 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:48.497467 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702f872-0f96-45a1-81d2-fc2a5a5df7dd None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:48.497944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e97bb2-9c23-4e0f-83c3-1a34595e00f8 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:48.499924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702f872-0f96-45a1-81d2-fc2a5a5df7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:48.501923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 234/457] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:48.505088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d219f0-7356-42f9-b436-3cb7d04ca30e None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:48.505591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d219f0-7356-42f9-b436-3cb7d04ca30e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:48.506907 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d219f0-7356-42f9-b436-3cb7d04ca30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:48.532092 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-4702f872-0f96-45a1-81d2-fc2a5a5df7dd None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:48.536123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4702f872-0f96-45a1-81d2-fc2a5a5df7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:48.546286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-a2d219f0-7356-42f9-b436-3cb7d04ca30e None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:48.550063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a2d219f0-7356-42f9-b436-3cb7d04ca30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:48.661315 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:48.675566 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 224/458] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:48.704962 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702f872-0f96-45a1-81d2-fc2a5a5df7dd None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:48.714868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 235/459] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:48.753467 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d219f0-7356-42f9-b436-3cb7d04ca30e None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:48.827080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:48.847745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702f872-0f96-45a1-81d2-fc2a5a5df7dd None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:48.884780 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d219f0-7356-42f9-b436-3cb7d04ca30e None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:48.889259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bba1ff-5343-4693-b7fb-b0ec881ff893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:48.890391 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bba1ff-5343-4693-b7fb-b0ec881ff893 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:48.890890 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bba1ff-5343-4693-b7fb-b0ec881ff893 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:48.891368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bba1ff-5343-4693-b7fb-b0ec881ff893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:48.903766 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bba1ff-5343-4693-b7fb-b0ec881ff893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:48.906010 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bba1ff-5343-4693-b7fb-b0ec881ff893 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:48.921652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f08f00-d488-4251-846e-d3c95babdaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:48.922665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f08f00-d488-4251-846e-d3c95babdaf2 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:48.923183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f08f00-d488-4251-846e-d3c95babdaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:48.923594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f08f00-d488-4251-846e-d3c95babdaf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:48.926951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-f0bba1ff-5343-4693-b7fb-b0ec881ff893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:48.926951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:48.926951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:48.926951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:48.926951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:48.926951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:48.926951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:48.926951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:48.926951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:48.926951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:48.926951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:48.926951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:48.926951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:52:48.926951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:52:48.926951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:52:48.927781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:52:48.927781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:48.927781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:48.927781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:48.927781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:48.927781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:48.927781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:48.927781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:48.927781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:48.927781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:48.927781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:48.927781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:48.927781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:48.927781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:48.927781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:48.927781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:48.927781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:48.928618 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:52:48.928618 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:52:48.928618 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:48.928618 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:48.931094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 225/460] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:48.932522 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8050db-3609-4ec2-9358-6e11176c58b1 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:48.933050 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8050db-3609-4ec2-9358-6e11176c58b1 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:48.933471 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8050db-3609-4ec2-9358-6e11176c58b1 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:48.935146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 226/461] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:48.936745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8050db-3609-4ec2-9358-6e11176c58b1 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:48.940082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f08f00-d488-4251-846e-d3c95babdaf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:48.942749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f08f00-d488-4251-846e-d3c95babdaf2 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:48.963240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-b9f08f00-d488-4251-846e-d3c95babdaf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:48.963240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:48.963240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:48.963240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:48.963240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:48.963240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:48.963240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:48.963240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:48.963240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:48.963240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:48.963240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:48.963240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:48.963240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:52:48.963240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:52:48.963240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:52:48.964123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:52:48.964123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:48.964123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:48.964123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:48.964123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:48.964123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:48.964123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:48.964123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:48.964123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:48.964123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:48.964123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:48.964123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:48.964123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:48.964123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:48.964123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:48.964123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:48.964123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:48.965107 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:52:48.965107 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:52:48.965107 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:48.965107 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:48.971547 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 236/462] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:48.973163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d09401-a1dc-46f8-89c6-af5d7f243c4a None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:48.973626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d09401-a1dc-46f8-89c6-af5d7f243c4a None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:48.974095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d09401-a1dc-46f8-89c6-af5d7f243c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:48.992103 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-21d09401-a1dc-46f8-89c6-af5d7f243c4a None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:48.994217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-21d09401-a1dc-46f8-89c6-af5d7f243c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:49.027554 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:49.058886 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8050db-3609-4ec2-9358-6e11176c58b1 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:49.084731 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 237/463] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:49.084731 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d09401-a1dc-46f8-89c6-af5d7f243c4a None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:49.094888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62677d-a1d1-4d60-8d76-12caca0770d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:49.095556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62677d-a1d1-4d60-8d76-12caca0770d2 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:49.095840 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62677d-a1d1-4d60-8d76-12caca0770d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:49.096118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62677d-a1d1-4d60-8d76-12caca0770d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:49.096881 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62677d-a1d1-4d60-8d76-12caca0770d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:49.097192 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62677d-a1d1-4d60-8d76-12caca0770d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:49.098573 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62677d-a1d1-4d60-8d76-12caca0770d2 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:49.106673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:49.112940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 227/464] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:49.114374 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65db80-2bbd-4bb6-ab82-e36a07e834ff None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:49.114669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65db80-2bbd-4bb6-ab82-e36a07e834ff None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:49.114950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65db80-2bbd-4bb6-ab82-e36a07e834ff None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:49.116319 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 228/465] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:49.121109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65db80-2bbd-4bb6-ab82-e36a07e834ff None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:49.203922 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d09401-a1dc-46f8-89c6-af5d7f243c4a None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:49.240935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65db80-2bbd-4bb6-ab82-e36a07e834ff None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:49.246709 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2afa7-4ac6-40f3-9604-6bbfcc4b0d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:49.247400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2afa7-4ac6-40f3-9604-6bbfcc4b0d12 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:49.247679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2afa7-4ac6-40f3-9604-6bbfcc4b0d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:49.247981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2afa7-4ac6-40f3-9604-6bbfcc4b0d12 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:49.248683 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2afa7-4ac6-40f3-9604-6bbfcc4b0d12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:49.249481 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2afa7-4ac6-40f3-9604-6bbfcc4b0d12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:49.250883 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2afa7-4ac6-40f3-9604-6bbfcc4b0d12 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:49.276119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 238/466] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:49.276119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2afa7-4ac6-40f3-9604-6bbfcc4b0d12 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:49.283465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ac36b-348f-4f89-8db9-4b6f3014cc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:49.284174 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ac36b-348f-4f89-8db9-4b6f3014cc8a None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:49.284464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ac36b-348f-4f89-8db9-4b6f3014cc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:49.284798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ac36b-348f-4f89-8db9-4b6f3014cc8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:49.296257 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ac36b-348f-4f89-8db9-4b6f3014cc8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:49.297772 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ac36b-348f-4f89-8db9-4b6f3014cc8a None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:49.303820 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:49.319316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-fb1ac36b-348f-4f89-8db9-4b6f3014cc8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:49.319316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:49.319316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:49.319316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:49.319316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:49.319316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:49.319316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:49.319316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:49.319316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:49.319316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:49.319316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:49.319316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:49.319316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:52:49.319316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:52:49.319316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:52:49.320210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:52:49.320210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:49.320210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:49.320210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:49.320210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:49.320210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:49.320210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:49.320210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:49.320210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:49.320210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:49.320210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:49.320210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:49.320210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:49.320210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:49.320210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:49.320210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:49.320210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:49.322236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:52:49.322236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:52:49.322236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:49.322236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:49.322236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 229/467] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:49.322759 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2856e-e538-4ae5-aed3-3b027fee7379 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:49.323047 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2856e-e538-4ae5-aed3-3b027fee7379 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:49.323324 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2856e-e538-4ae5-aed3-3b027fee7379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:49.340773 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-38b2856e-e538-4ae5-aed3-3b027fee7379 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:49.341984 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-38b2856e-e538-4ae5-aed3-3b027fee7379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:49.389908 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2afa7-4ac6-40f3-9604-6bbfcc4b0d12 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:49.391538 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:49.421089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 230/468] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:49.421089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2856e-e538-4ae5-aed3-3b027fee7379 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:49.432892 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdb6b9-52e7-41b2-a91b-b704fa3ddb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:49.433590 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdb6b9-52e7-41b2-a91b-b704fa3ddb4c None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:49.433877 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdb6b9-52e7-41b2-a91b-b704fa3ddb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:49.434159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdb6b9-52e7-41b2-a91b-b704fa3ddb4c None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:49.434873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdb6b9-52e7-41b2-a91b-b704fa3ddb4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:49.436449 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdb6b9-52e7-41b2-a91b-b704fa3ddb4c None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:49.458208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 239/469] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:52:49 2022] POST /identity/v3/users => generated 248 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:52:49.469813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdb6b9-52e7-41b2-a91b-b704fa3ddb4c None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:49.560742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2856e-e538-4ae5-aed3-3b027fee7379 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:49.585952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:49.609823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468581d-3162-4239-aa52-53cb724e18ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:49.610606 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468581d-3162-4239-aa52-53cb724e18ac None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:49.610950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468581d-3162-4239-aa52-53cb724e18ac None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:49.611569 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468581d-3162-4239-aa52-53cb724e18ac None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:49.612421 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468581d-3162-4239-aa52-53cb724e18ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:49.614077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468581d-3162-4239-aa52-53cb724e18ac None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:49.615964 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdb6b9-52e7-41b2-a91b-b704fa3ddb4c None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:49.638285 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 231/470] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:52:49 2022] POST /identity/v3/users => generated 246 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:52:49.639168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468581d-3162-4239-aa52-53cb724e18ac None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:49.658828 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3708124-463d-44e9-81dc-2f39724fd9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:49.659945 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3708124-463d-44e9-81dc-2f39724fd9f8 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:49.660564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3708124-463d-44e9-81dc-2f39724fd9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:49.660898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3708124-463d-44e9-81dc-2f39724fd9f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:49.661660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3708124-463d-44e9-81dc-2f39724fd9f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:49.662033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3708124-463d-44e9-81dc-2f39724fd9f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:49.663492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3708124-463d-44e9-81dc-2f39724fd9f8 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:49.682374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 240/471] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:49.689890 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3708124-463d-44e9-81dc-2f39724fd9f8 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:49.690649 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:49.790660 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468581d-3162-4239-aa52-53cb724e18ac None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:49.825917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3708124-463d-44e9-81dc-2f39724fd9f8 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:49.838319 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8233f-12cc-43a2-8562-b6da85d8f148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:49.839458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8233f-12cc-43a2-8562-b6da85d8f148 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:49.839746 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8233f-12cc-43a2-8562-b6da85d8f148 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:49.840023 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8233f-12cc-43a2-8562-b6da85d8f148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:49.852744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8233f-12cc-43a2-8562-b6da85d8f148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:49.854379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8233f-12cc-43a2-8562-b6da85d8f148 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:49.867406 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b95a0-ad62-49c5-8b63-362e456c78ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:49.869545 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b95a0-ad62-49c5-8b63-362e456c78ea None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:49.869545 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b95a0-ad62-49c5-8b63-362e456c78ea None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:49.869545 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b95a0-ad62-49c5-8b63-362e456c78ea None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:49.870222 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b95a0-ad62-49c5-8b63-362e456c78ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:49.871648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b95a0-ad62-49c5-8b63-362e456c78ea None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:49.873642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-36d8233f-12cc-43a2-8562-b6da85d8f148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:49.873642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:49.873642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:49.873642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:49.873642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:49.873642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:49.873642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:49.873642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:49.873642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:49.873642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:49.873642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:49.873642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:49.873642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:52:49.873642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:52:49.873642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:52:49.874490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:52:49.874490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:49.874490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:49.874490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:49.874490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:49.874490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:49.874490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:49.874490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:49.874490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:49.874490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:49.874490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:49.874490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:49.874490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:49.874490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:49.874490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:49.874490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:49.874490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:49.881423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:52:49.881423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:52:49.881423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:49.881423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:49.881423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 232/472] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:49.881423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39355bc-b117-4056-8815-082a5f3f80bc None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:49.881423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39355bc-b117-4056-8815-082a5f3f80bc None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:49.881423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39355bc-b117-4056-8815-082a5f3f80bc None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:49.881423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 233/473] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:49.885729 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39355bc-b117-4056-8815-082a5f3f80bc None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:49.896269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:49.912749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 241/474] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:52:49 2022] POST /identity/v3/users => generated 248 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:52:49.912749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324cc6a-fcc6-4bf9-a785-cc3d3841d7be None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:49.912749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324cc6a-fcc6-4bf9-a785-cc3d3841d7be None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:49.912749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324cc6a-fcc6-4bf9-a785-cc3d3841d7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:49.938211 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-9324cc6a-fcc6-4bf9-a785-cc3d3841d7be None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:49.939815 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-9324cc6a-fcc6-4bf9-a785-cc3d3841d7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:50.037167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:50.053069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 242/475] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:50.059800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39355bc-b117-4056-8815-082a5f3f80bc None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:50.091585 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324cc6a-fcc6-4bf9-a785-cc3d3841d7be None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:50.113317 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5e5a5-60f3-4806-95bf-95d6a9a24b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:50.113999 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5e5a5-60f3-4806-95bf-95d6a9a24b4b None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:50.114283 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5e5a5-60f3-4806-95bf-95d6a9a24b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:50.114562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5e5a5-60f3-4806-95bf-95d6a9a24b4b None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:50.115294 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c5e5a5-60f3-4806-95bf-95d6a9a24b4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:50.115602 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c5e5a5-60f3-4806-95bf-95d6a9a24b4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:50.117025 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c5e5a5-60f3-4806-95bf-95d6a9a24b4b None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:50.133040 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 234/476] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:50.140310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5e5a5-60f3-4806-95bf-95d6a9a24b4b None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:50.230681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324cc6a-fcc6-4bf9-a785-cc3d3841d7be None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:50.264584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:50.273712 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5e5a5-60f3-4806-95bf-95d6a9a24b4b None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:50.274586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3144d3-bd91-477a-88f6-b3d9f78a18b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:50.275780 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3144d3-bd91-477a-88f6-b3d9f78a18b3 None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:50.276067 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3144d3-bd91-477a-88f6-b3d9f78a18b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:50.276343 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3144d3-bd91-477a-88f6-b3d9f78a18b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:50.277263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3144d3-bd91-477a-88f6-b3d9f78a18b3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:50.278664 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3144d3-bd91-477a-88f6-b3d9f78a18b3 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:50.287082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.sql.core [None req-fd3144d3-bd91-477a-88f6-b3d9f78a18b3 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 21:52:50.287082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 12 21:52:50.287082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [parameters: {'id': '06f96b3e9362411d8a5a7a67ee3cff18', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 12 21:52:50.287082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80218) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 21:52:50.307485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-fd3144d3-bd91-477a-88f6-b3d9f78a18b3 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 12 21:52:50.307485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:50.307485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 21:52:50.307485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 21:52:50.307485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 21:52:50.307485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 21:52:50.307485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 21:52:50.307485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 21:52:50.307485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 21:52:50.307485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application conn.query(q) Jul 12 21:52:50.307485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 21:52:50.307485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 21:52:50.307485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 21:52:50.307485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application result.read() Jul 12 21:52:50.307485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return ref.to_dict() Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 21:52:50.308400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application next(self.gen) Jul 12 21:52:50.309364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 21:52:50.309364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application yield resource Jul 12 21:52:50.309364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 21:52:50.309364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application next(self.gen) Jul 12 21:52:50.309364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 21:52:50.309364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 21:52:50.309364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 21:52:50.309364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 21:52:50.309364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 21:52:50.309364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise self.value Jul 12 21:52:50.309364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 21:52:50.309364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 21:52:50.309364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 21:52:50.309364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application session.commit() Jul 12 21:52:50.309364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 21:52:50.309364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 21:52:50.309364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 21:52:50.311838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 21:52:50.311838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 21:52:50.311838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self.session.flush() Jul 12 21:52:50.311838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 21:52:50.311838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 21:52:50.311838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 21:52:50.311838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 21:52:50.311838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 21:52:50.311838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application compat.raise_( Jul 12 21:52:50.311838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 21:52:50.311838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception Jul 12 21:52:50.311838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 21:52:50.311838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 21:52:50.311838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 21:52:50.311838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 21:52:50.311838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 21:52:50.311838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 21:52:50.320756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 21:52:50.320756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 21:52:50.320756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 12 21:52:50.320756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 12 21:52:50.320756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 21:52:50.320756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 21:52:50.320756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 21:52:50.320756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 21:52:50.320756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 21:52:50.320756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 21:52:50.320756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 21:52:50.320756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 21:52:50.320756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 21:52:50.320756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 21:52:50.320756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 21:52:50.320756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception Jul 12 21:52:50.322788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 21:52:50.322788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 21:52:50.322788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 21:52:50.322788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 21:52:50.322788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 21:52:50.322788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 21:52:50.322788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 21:52:50.322788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application conn.query(q) Jul 12 21:52:50.322788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 21:52:50.322788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 21:52:50.322788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 21:52:50.322788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application result.read() Jul 12 21:52:50.322788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 21:52:50.322788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 21:52:50.322788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 21:52:50.322788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 21:52:50.324648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 21:52:50.324648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 21:52:50.324648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 21:52:50.324648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 21:52:50.324648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 21:52:50.324648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 12 21:52:50.324648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [parameters: {'id': '06f96b3e9362411d8a5a7a67ee3cff18', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 12 21:52:50.324648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 21:52:50.324648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:50.324648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:52:50.324648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:50.324648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:50.324648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:50.324648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:50.324648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:50.324648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:50.326665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:50.326665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:50.326665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:50.326665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:50.326665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:50.326665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:50.326665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 12 21:52:50.326665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 12 21:52:50.326665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:50.326665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:50.326665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 12 21:52:50.326665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 12 21:52:50.326665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 21:52:50.326665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 21:52:50.326665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 12 21:52:50.326665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:50.328133 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 243/477] 10.222.0.44 () {60 vars in 1131 bytes} [Tue Jul 12 21:52:50 2022] POST /identity/v3/roles => generated 144 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:52:50.328257 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76636f56-8e36-4e71-a250-4be27f07c7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:50.328257 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76636f56-8e36-4e71-a250-4be27f07c7c8 None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:50.328257 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76636f56-8e36-4e71-a250-4be27f07c7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:50.328257 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76636f56-8e36-4e71-a250-4be27f07c7c8 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:50.328701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76636f56-8e36-4e71-a250-4be27f07c7c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:50.328846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3144d3-bd91-477a-88f6-b3d9f78a18b3 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:50.328981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76636f56-8e36-4e71-a250-4be27f07c7c8 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:50.350333 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 235/478] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:52:50 2022] POST /identity/v3/users => generated 249 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 21:52:50.450755 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3144d3-bd91-477a-88f6-b3d9f78a18b3 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:50.489461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddaf937-5a00-4e31-9566-ae92f4cf3d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:50.490342 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddaf937-5a00-4e31-9566-ae92f4cf3d61 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:50.490626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddaf937-5a00-4e31-9566-ae92f4cf3d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:50.490901 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddaf937-5a00-4e31-9566-ae92f4cf3d61 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:50.503103 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddaf937-5a00-4e31-9566-ae92f4cf3d61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:50.511990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-cddaf937-5a00-4e31-9566-ae92f4cf3d61 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:50.511990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:50.511990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:50.511990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:50.511990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:50.511990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:50.511990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:50.511990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:50.511990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:50.511990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:50.511990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:50.511990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:50.511990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:50.511990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:50.511990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:50.511990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:50.513068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:50.513068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:50.513068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:50.513068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:50.513068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:50.513068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:50.513068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:50.513068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:50.513068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:50.513068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:50.513068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:50.513068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:50.513068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:50.513068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:50.513068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:50.513068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:50.513882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:50.513882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:50.513882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:50.513882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:50.513882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:50.513882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:50.513882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:50.513882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:50.513882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:50.513882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:50.513882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 21:52:50.513882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:50.513882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 244/479] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:52:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:50.522079 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76636f56-8e36-4e71-a250-4be27f07c7c8 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:50.530082 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:50.648026 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76636f56-8e36-4e71-a250-4be27f07c7c8 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:50.686526 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93a704-cb01-4ccf-9552-f3cce6247edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:50.687210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93a704-cb01-4ccf-9552-f3cce6247edf None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:50.687489 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93a704-cb01-4ccf-9552-f3cce6247edf None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:50.687761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93a704-cb01-4ccf-9552-f3cce6247edf None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:50.688506 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93a704-cb01-4ccf-9552-f3cce6247edf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:50.688894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93a704-cb01-4ccf-9552-f3cce6247edf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:50.690268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93a704-cb01-4ccf-9552-f3cce6247edf None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:50.700614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 236/480] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:52:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:51.797342 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453544f-a580-4d0a-8bf1-b899f55af305 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:51.797863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453544f-a580-4d0a-8bf1-b899f55af305 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:51.797937 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453544f-a580-4d0a-8bf1-b899f55af305 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:51.799486 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 245/481] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:51.805629 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd80aee-f065-462a-9010-82929c8d6f96 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:51.806445 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd80aee-f065-462a-9010-82929c8d6f96 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:51.806917 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd80aee-f065-462a-9010-82929c8d6f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:51.824623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-0fd80aee-f065-462a-9010-82929c8d6f96 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:51.826164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0fd80aee-f065-462a-9010-82929c8d6f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:51.913375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:51.927161 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 237/482] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:52:51.979087 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453544f-a580-4d0a-8bf1-b899f55af305 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:52.071813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:52.140739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453544f-a580-4d0a-8bf1-b899f55af305 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:52.193849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921822c8-6b41-4595-8350-233f58b84ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:52.194665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921822c8-6b41-4595-8350-233f58b84ced None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:52.194949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921822c8-6b41-4595-8350-233f58b84ced None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:52.195226 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921822c8-6b41-4595-8350-233f58b84ced None admin] PATH_INFO: `/v3/roles/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:52.212384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921822c8-6b41-4595-8350-233f58b84ced None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:52.220159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-921822c8-6b41-4595-8350-233f58b84ced None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:52:52.220159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:52.220159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:52.220159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:52.220159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:52.220159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:52.220159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:52.220159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:52.220159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:52.220159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:52.220159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:52.220159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:52.220159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:52.220159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:52.220159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:52.220159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:52.221043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:52.221043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:52.221043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:52.221043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:52.221043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:52.221043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:52.221043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:52.221043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:52.221043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:52.221043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:52.221043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:52.221043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:52.221043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:52.221043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:52.221043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:52.221043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:52.222256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:52.222256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:52.222256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:52.222256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:52.222256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:52.222256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:52.222256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:52.222256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:52.222256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:52.222256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:52.222256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:52:52.222256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:52.222256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 246/483] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:52:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:52.229480 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd80aee-f065-462a-9010-82929c8d6f96 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:52.346706 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd80aee-f065-462a-9010-82929c8d6f96 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:52.388806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299f64c-d81c-4c13-9f68-292584f59c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:52.389835 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299f64c-d81c-4c13-9f68-292584f59c82 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:52.390623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299f64c-d81c-4c13-9f68-292584f59c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:52.391400 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299f64c-d81c-4c13-9f68-292584f59c82 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:52.392376 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299f64c-d81c-4c13-9f68-292584f59c82 None admin] RBAC: Adding query filter params (name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:52.392920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299f64c-d81c-4c13-9f68-292584f59c82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:52.394755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299f64c-d81c-4c13-9f68-292584f59c82 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:52.409163 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 238/484] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:52:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:52.416254 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921822c8-6b41-4595-8350-233f58b84ced None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:52.569149 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921822c8-6b41-4595-8350-233f58b84ced None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:52.591476 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:52.618008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f2ae0-c57c-4eba-aef8-a87f704b9086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:52.618766 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f2ae0-c57c-4eba-aef8-a87f704b9086 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:52.619122 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f2ae0-c57c-4eba-aef8-a87f704b9086 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:52.619403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f2ae0-c57c-4eba-aef8-a87f704b9086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:52.634457 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb20125-9838-4ed7-80ea-8ce209c3816c None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:52.635534 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb20125-9838-4ed7-80ea-8ce209c3816c None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:52.636019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb20125-9838-4ed7-80ea-8ce209c3816c None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:52.637404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f2ae0-c57c-4eba-aef8-a87f704b9086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:52.639493 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f2ae0-c57c-4eba-aef8-a87f704b9086 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:52.639783 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 239/485] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:52.645111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0593c-3342-4405-bffc-c3d3997337d7 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:52.645653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0593c-3342-4405-bffc-c3d3997337d7 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:52.646124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0593c-3342-4405-bffc-c3d3997337d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:52.664036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-ab7f2ae0-c57c-4eba-aef8-a87f704b9086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:52.664036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:52.664036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:52.664036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:52.664036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:52.664036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:52.664036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:52.664036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:52.664036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:52.664036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:52.664036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:52.664036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:52.664036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:52:52.664036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:52:52.664036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:52:52.665069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:52:52.665069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:52.665069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:52.665069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:52.665069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:52.665069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:52.665069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:52.665069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:52.665069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:52.665069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:52.665069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:52.665069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:52.665069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:52.665069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:52.665069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:52.665069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:52.665069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:52.665915 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:52:52.665915 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:52:52.665915 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:52.665915 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:52.672809 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 247/486] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:52.675134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f2ae0-c57c-4eba-aef8-a87f704b9086 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:52.685662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-9cd0593c-3342-4405-bffc-c3d3997337d7 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:52.686870 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-9cd0593c-3342-4405-bffc-c3d3997337d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:52.744184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:52.763436 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 240/487] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:52:52.763436 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856f603-7fbf-4484-888e-32b52b7865af None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:52.763436 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856f603-7fbf-4484-888e-32b52b7865af None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:52.763436 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856f603-7fbf-4484-888e-32b52b7865af None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:52.763436 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 241/488] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:52.777502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943d264-c292-4cad-b651-2d83686a2cbc None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:52.777796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943d264-c292-4cad-b651-2d83686a2cbc None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:52.778077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943d264-c292-4cad-b651-2d83686a2cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:52.805702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-0943d264-c292-4cad-b651-2d83686a2cbc None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:52.807088 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0943d264-c292-4cad-b651-2d83686a2cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:52.818252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f2ae0-c57c-4eba-aef8-a87f704b9086 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:52.864947 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee248f-827b-4255-ae4d-799cfd1f0c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:52.865965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee248f-827b-4255-ae4d-799cfd1f0c3d None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:52.866442 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee248f-827b-4255-ae4d-799cfd1f0c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:52.866930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee248f-827b-4255-ae4d-799cfd1f0c3d None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:52.868938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee248f-827b-4255-ae4d-799cfd1f0c3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:52.869432 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee248f-827b-4255-ae4d-799cfd1f0c3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:52.871686 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee248f-827b-4255-ae4d-799cfd1f0c3d None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:52.889187 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 248/489] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:52.890420 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee248f-827b-4255-ae4d-799cfd1f0c3d None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:52.907736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 242/490] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:52.913003 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943d264-c292-4cad-b651-2d83686a2cbc None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:52.937344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:53.048269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee248f-827b-4255-ae4d-799cfd1f0c3d None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:53.054339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943d264-c292-4cad-b651-2d83686a2cbc None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:53.074752 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:53.087782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af7bd8-6c71-44ee-b0f5-9b2aaea5df77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:53.088594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af7bd8-6c71-44ee-b0f5-9b2aaea5df77 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:53.088914 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af7bd8-6c71-44ee-b0f5-9b2aaea5df77 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:53.089197 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af7bd8-6c71-44ee-b0f5-9b2aaea5df77 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:53.093281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073932e7-bf31-4098-a0c2-793c8d0379b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:53.094079 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073932e7-bf31-4098-a0c2-793c8d0379b8 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:53.094368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073932e7-bf31-4098-a0c2-793c8d0379b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:53.094631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073932e7-bf31-4098-a0c2-793c8d0379b8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:53.100486 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af7bd8-6c71-44ee-b0f5-9b2aaea5df77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:53.102628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073932e7-bf31-4098-a0c2-793c8d0379b8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:53.104090 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073932e7-bf31-4098-a0c2-793c8d0379b8 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:53.109233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-69af7bd8-6c71-44ee-b0f5-9b2aaea5df77 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:52:53.109233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:53.109233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:53.109233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:53.109233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:53.109233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:53.109233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:53.109233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:53.109233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:53.109233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:53.109233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:53.109233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:53.109233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:53.109233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:53.109233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:53.109233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:53.110090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:53.110090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:53.110090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:53.110090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:53.110090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:53.110090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:53.110090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:53.110090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:53.110090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:53.110090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:53.110090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:53.110090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:53.110090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:53.110090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:53.110090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:53.110090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:53.110873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:53.110873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:53.110873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:53.110873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:53.110873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:53.110873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:53.110873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:53.110873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:53.110873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:53.110873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:53.110873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:52:53.110873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:53.110873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 249/491] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:52:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:53.111510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af7bd8-6c71-44ee-b0f5-9b2aaea5df77 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:53.125305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-073932e7-bf31-4098-a0c2-793c8d0379b8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 21:52:53.125305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:53.125305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:53.125305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:53.125305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:53.125305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:53.125305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:53.125305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:53.125305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:53.125305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:53.125305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:53.125305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:53.125305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:52:53.125305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:52:53.125305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:52:53.125305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:53.126274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:53.127167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:53.127167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:53.127167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:52:53.127167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:52:53.127167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:52:53.127167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:52:53.127167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:52:53.127167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:52:53.127167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 21:52:53.127167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:53.127167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 243/492] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:53.128619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d856cdf-b6b9-4e17-836a-f7770c195bbb None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:53.128929 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d856cdf-b6b9-4e17-836a-f7770c195bbb None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:53.129209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d856cdf-b6b9-4e17-836a-f7770c195bbb None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:53.130964 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 244/493] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:53.131770 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d856cdf-b6b9-4e17-836a-f7770c195bbb None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:53.208132 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:53.226162 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af7bd8-6c71-44ee-b0f5-9b2aaea5df77 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:53.245971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d856cdf-b6b9-4e17-836a-f7770c195bbb None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:53.264181 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec5cc6-dd55-40da-a5bc-40ed8a18e55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:53.265007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec5cc6-dd55-40da-a5bc-40ed8a18e55b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:53.265289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec5cc6-dd55-40da-a5bc-40ed8a18e55b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:53.265565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec5cc6-dd55-40da-a5bc-40ed8a18e55b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:53.277020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec5cc6-dd55-40da-a5bc-40ed8a18e55b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:53.282097 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf28cee-b3fc-4f72-b1e0-2b7984522d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:53.282768 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf28cee-b3fc-4f72-b1e0-2b7984522d23 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:53.283057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf28cee-b3fc-4f72-b1e0-2b7984522d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:53.283335 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf28cee-b3fc-4f72-b1e0-2b7984522d23 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:53.284079 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf28cee-b3fc-4f72-b1e0-2b7984522d23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:53.284390 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf28cee-b3fc-4f72-b1e0-2b7984522d23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:53.284787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-04ec5cc6-dd55-40da-a5bc-40ed8a18e55b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:52:53.284787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:53.284787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:53.284787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:53.284787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:53.284787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:53.284787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:53.284787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:53.284787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:53.284787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:53.284787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:53.284787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:53.284787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:53.284787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:53.284787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:53.284787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:53.285665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:53.285665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:53.285665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:53.285665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:53.285665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:53.285665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:53.285665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:53.285665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:53.285665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:53.285665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:53.285665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:53.285665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:53.285665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:53.285665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:53.285665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:53.285665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:53.286469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:53.286469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:53.286469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:53.286469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:53.286469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:53.286469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:53.286469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:53.286469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:53.286469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:53.286469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:53.286469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:52:53.286469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:53.287096 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf28cee-b3fc-4f72-b1e0-2b7984522d23 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:53.291031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 250/494] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:52:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:53.292480 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314869bf-c15a-4243-967c-3503bb445719 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:53.292586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314869bf-c15a-4243-967c-3503bb445719 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:53.292915 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314869bf-c15a-4243-967c-3503bb445719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:53.300038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 245/495] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:52:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:53.300038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf28cee-b3fc-4f72-b1e0-2b7984522d23 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:53.315322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-314869bf-c15a-4243-967c-3503bb445719 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:53.316796 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-314869bf-c15a-4243-967c-3503bb445719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:53.342664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:53.389080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 251/496] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:53.390222 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314869bf-c15a-4243-967c-3503bb445719 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:53.419649 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf28cee-b3fc-4f72-b1e0-2b7984522d23 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:53.456414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a525ee4-a99b-446c-a3e1-861a29292365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:53.457040 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a525ee4-a99b-446c-a3e1-861a29292365 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:53.457315 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a525ee4-a99b-446c-a3e1-861a29292365 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:53.457590 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a525ee4-a99b-446c-a3e1-861a29292365 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:53.458427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a525ee4-a99b-446c-a3e1-861a29292365 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:53.458736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a525ee4-a99b-446c-a3e1-861a29292365 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:53.460171 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a525ee4-a99b-446c-a3e1-861a29292365 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:53.479354 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:53.498471 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 246/497] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Jul 12 21:52:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:53.499392 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a525ee4-a99b-446c-a3e1-861a29292365 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:53.505597 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314869bf-c15a-4243-967c-3503bb445719 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:53.541271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f73e87-7117-4180-865b-452b8d5ad44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:53.541936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f73e87-7117-4180-865b-452b8d5ad44e None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:53.542217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f73e87-7117-4180-865b-452b8d5ad44e None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:53.542486 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f73e87-7117-4180-865b-452b8d5ad44e None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:53.543227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f73e87-7117-4180-865b-452b8d5ad44e None admin] RBAC: Adding query filter params (name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:53.543529 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f73e87-7117-4180-865b-452b8d5ad44e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:53.544938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f73e87-7117-4180-865b-452b8d5ad44e None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:53.552873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 252/498] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:52:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:53.553681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f73e87-7117-4180-865b-452b8d5ad44e None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:53.605068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:53.609037 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a525ee4-a99b-446c-a3e1-861a29292365 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:53.645466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a167d0a-9a2e-46ca-8dfe-d380fb54d50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:53.646184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a167d0a-9a2e-46ca-8dfe-d380fb54d50d None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:53.646467 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a167d0a-9a2e-46ca-8dfe-d380fb54d50d None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:53.646739 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a167d0a-9a2e-46ca-8dfe-d380fb54d50d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:53.652093 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f73e87-7117-4180-865b-452b8d5ad44e None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:53.657016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a167d0a-9a2e-46ca-8dfe-d380fb54d50d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:53.658492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a167d0a-9a2e-46ca-8dfe-d380fb54d50d None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:53.676375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-6a167d0a-9a2e-46ca-8dfe-d380fb54d50d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:53.676375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:53.676375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:53.676375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:53.676375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:53.676375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:53.676375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:53.676375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:53.676375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:53.676375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:53.676375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:53.676375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:53.676375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:52:53.676375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:52:53.676375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:52:53.677284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:52:53.677284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:53.677284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:53.677284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:53.677284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:53.677284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:53.677284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:53.677284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:53.677284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:53.677284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:53.677284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:53.677284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:53.677284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:53.677284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:53.677284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:53.677284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:53.677284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:53.678149 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:52:53.678149 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:52:53.678149 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:53.678149 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:53.678149 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 247/499] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:53.678804 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab1e6e-cf9f-4c95-b9c9-0f9af77c36fa None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:53.679070 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab1e6e-cf9f-4c95-b9c9-0f9af77c36fa None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:53.679346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab1e6e-cf9f-4c95-b9c9-0f9af77c36fa None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:53.681435 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 248/500] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:53.681969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab1e6e-cf9f-4c95-b9c9-0f9af77c36fa None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:53.692066 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da57d4-b6c1-4256-9d85-24dc832a59c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:53.692887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da57d4-b6c1-4256-9d85-24dc832a59c3 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:53.693169 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da57d4-b6c1-4256-9d85-24dc832a59c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:53.693444 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da57d4-b6c1-4256-9d85-24dc832a59c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:53.696565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:53.704903 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da57d4-b6c1-4256-9d85-24dc832a59c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:53.712594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-30da57d4-b6c1-4256-9d85-24dc832a59c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:52:53.712594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:53.712594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:53.712594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:53.712594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:53.712594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:53.712594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:53.712594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:53.712594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:53.712594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:53.712594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:53.712594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:53.712594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:53.712594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:53.712594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:53.712594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:53.713695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:53.713695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:53.713695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:53.713695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:53.713695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:53.713695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:53.713695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:53.713695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:53.713695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:53.713695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:53.713695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:53.713695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:53.713695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:53.713695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:53.713695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:53.713695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:53.714680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:53.714680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:53.714680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:53.714680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:53.714680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:53.714680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:53.714680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:53.714680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:53.714680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:53.714680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:53.714680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:52:53.714680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:53.714680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 253/501] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:52:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:53.714680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da57d4-b6c1-4256-9d85-24dc832a59c3 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:53.792205 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab1e6e-cf9f-4c95-b9c9-0f9af77c36fa None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:53.812592 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da57d4-b6c1-4256-9d85-24dc832a59c3 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:53.825929 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc42ab-e985-45ee-b3dd-8f07c00af2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:53.826588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc42ab-e985-45ee-b3dd-8f07c00af2ca None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:53.826878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc42ab-e985-45ee-b3dd-8f07c00af2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:53.827155 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc42ab-e985-45ee-b3dd-8f07c00af2ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:53.845650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:53.846601 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732e8f8-eb4d-4881-b087-8289b59f5051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:53.847290 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732e8f8-eb4d-4881-b087-8289b59f5051 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:53.847540 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732e8f8-eb4d-4881-b087-8289b59f5051 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:53.847857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732e8f8-eb4d-4881-b087-8289b59f5051 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:53.859419 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732e8f8-eb4d-4881-b087-8289b59f5051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:53.860921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732e8f8-eb4d-4881-b087-8289b59f5051 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:53.878070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-5732e8f8-eb4d-4881-b087-8289b59f5051 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:53.878070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:53.878070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:53.878070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:53.878070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:53.878070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:53.878070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:53.878070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:53.878070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:53.878070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:53.878070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:53.878070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:53.878070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:52:53.878070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:52:53.878070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:52:53.878976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:52:53.878976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:53.878976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:53.878976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:53.878976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:53.878976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:53.878976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:53.878976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:53.878976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:53.878976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:53.878976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:53.878976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:53.878976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:53.878976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:53.878976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:53.878976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:53.878976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:53.879832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:52:53.879832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:52:53.879832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:53.879832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:53.879832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 254/502] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:53.880098 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732e8f8-eb4d-4881-b087-8289b59f5051 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:53.902253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc42ab-e985-45ee-b3dd-8f07c00af2ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:53.903638 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc42ab-e985-45ee-b3dd-8f07c00af2ca None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:53.934823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:53.987955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732e8f8-eb4d-4881-b087-8289b59f5051 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:54.007789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 249/503] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:54.010076 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f56f86-b4a8-4155-9749-309e669f6e22 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:54.010369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f56f86-b4a8-4155-9749-309e669f6e22 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:54.010928 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f56f86-b4a8-4155-9749-309e669f6e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:54.023858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05d4fe-449a-43d3-afcb-cf2258eb969e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:54.024677 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05d4fe-449a-43d3-afcb-cf2258eb969e None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:54.024992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05d4fe-449a-43d3-afcb-cf2258eb969e None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:54.025269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05d4fe-449a-43d3-afcb-cf2258eb969e None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:54.025999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05d4fe-449a-43d3-afcb-cf2258eb969e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:54.026312 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05d4fe-449a-43d3-afcb-cf2258eb969e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:54.026880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-05f56f86-b4a8-4155-9749-309e669f6e22 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:54.027686 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05d4fe-449a-43d3-afcb-cf2258eb969e None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:54.028080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-05f56f86-b4a8-4155-9749-309e669f6e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:54.040079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 255/504] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:54.040936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05d4fe-449a-43d3-afcb-cf2258eb969e None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:54.074455 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:54.096019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 250/505] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:54.096881 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f56f86-b4a8-4155-9749-309e669f6e22 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:54.146553 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05d4fe-449a-43d3-afcb-cf2258eb969e None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:54.155999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:54.181279 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a7d7c-b7e0-4229-997f-52e6da5ef84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:54.181945 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a7d7c-b7e0-4229-997f-52e6da5ef84a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:54.182229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a7d7c-b7e0-4229-997f-52e6da5ef84a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:54.182507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a7d7c-b7e0-4229-997f-52e6da5ef84a None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:54.183254 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a7d7c-b7e0-4229-997f-52e6da5ef84a None admin] RBAC: Adding query filter params (name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:54.183565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a7d7c-b7e0-4229-997f-52e6da5ef84a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:54.184976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a7d7c-b7e0-4229-997f-52e6da5ef84a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:54.193490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 256/506] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:52:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:54.194141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a7d7c-b7e0-4229-997f-52e6da5ef84a None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:54.214606 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f56f86-b4a8-4155-9749-309e669f6e22 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:54.249042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d9cec-7d7e-4fd4-8aeb-fc7acdfa5b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:54.249926 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d9cec-7d7e-4fd4-8aeb-fc7acdfa5b6c None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:54.250208 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d9cec-7d7e-4fd4-8aeb-fc7acdfa5b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:54.250485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d9cec-7d7e-4fd4-8aeb-fc7acdfa5b6c None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:54.251207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d9cec-7d7e-4fd4-8aeb-fc7acdfa5b6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:54.251519 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d9cec-7d7e-4fd4-8aeb-fc7acdfa5b6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:54.252920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d9cec-7d7e-4fd4-8aeb-fc7acdfa5b6c None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:54.264768 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 251/507] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:54.265579 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d9cec-7d7e-4fd4-8aeb-fc7acdfa5b6c None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:54.294179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a7d7c-b7e0-4229-997f-52e6da5ef84a None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:54.309191 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:54.328426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e135d11-feaf-4fad-b5d8-7a3f6a40b984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:54.329246 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e135d11-feaf-4fad-b5d8-7a3f6a40b984 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:54.329526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e135d11-feaf-4fad-b5d8-7a3f6a40b984 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:54.329805 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e135d11-feaf-4fad-b5d8-7a3f6a40b984 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:54.340693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e135d11-feaf-4fad-b5d8-7a3f6a40b984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:54.342170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e135d11-feaf-4fad-b5d8-7a3f6a40b984 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:54.359128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-3e135d11-feaf-4fad-b5d8-7a3f6a40b984 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:54.359128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:54.359128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:54.359128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:54.359128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:54.359128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:54.359128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:54.359128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:54.359128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:54.359128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:54.359128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:54.359128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:54.359128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:52:54.359128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:52:54.359128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:52:54.359981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:52:54.359981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:54.359981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:54.359981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:54.359981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:54.359981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:54.359981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:54.359981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:54.359981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:54.359981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:54.359981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:54.359981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:54.359981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:54.359981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:54.359981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:54.359981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:54.359981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:54.360854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:52:54.360854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:52:54.360854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:54.360854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:54.360854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 257/508] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:54.361125 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e135d11-feaf-4fad-b5d8-7a3f6a40b984 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:54.374169 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d9cec-7d7e-4fd4-8aeb-fc7acdfa5b6c None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:54.387100 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:54.410066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9c2bf-30c7-491d-9ec9-d8b1dbd2b0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:54.410805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9c2bf-30c7-491d-9ec9-d8b1dbd2b0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:54.411074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9c2bf-30c7-491d-9ec9-d8b1dbd2b0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:54.411349 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9c2bf-30c7-491d-9ec9-d8b1dbd2b0c1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:54.419263 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9c2bf-30c7-491d-9ec9-d8b1dbd2b0c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:54.420740 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9c2bf-30c7-491d-9ec9-d8b1dbd2b0c1 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:54.435036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-62b9c2bf-30c7-491d-9ec9-d8b1dbd2b0c1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 21:52:54.435036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:54.435036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:54.435036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:54.435036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:54.435036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:54.435036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:54.435036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:54.435036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:54.435036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:54.435036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:54.435036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:54.435036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:52:54.435036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:52:54.435036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:52:54.435036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:54.435969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:54.436922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:54.436922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:54.436922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:52:54.436922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:52:54.436922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:52:54.436922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:52:54.436922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:52:54.436922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:52:54.436922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 21:52:54.436922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:54.436922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 252/509] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:54.437492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9c2bf-30c7-491d-9ec9-d8b1dbd2b0c1 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:54.463633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e135d11-feaf-4fad-b5d8-7a3f6a40b984 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:54.498298 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d0cc5-d267-4ccf-acfc-05b88cd3712d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:54.499087 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d0cc5-d267-4ccf-acfc-05b88cd3712d None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:54.499369 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d0cc5-d267-4ccf-acfc-05b88cd3712d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:54.499645 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d0cc5-d267-4ccf-acfc-05b88cd3712d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:54.510129 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d0cc5-d267-4ccf-acfc-05b88cd3712d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:54.511610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d0cc5-d267-4ccf-acfc-05b88cd3712d None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:54.528863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-319d0cc5-d267-4ccf-acfc-05b88cd3712d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:54.528863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:54.528863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:54.528863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:54.528863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:54.528863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:54.528863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:54.528863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:54.528863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:54.528863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:54.528863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:54.528863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:54.528863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:52:54.528863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:52:54.528863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:52:54.529848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:52:54.529848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:54.529848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:54.529848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:54.529848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:54.529848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:54.529848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:54.529848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:54.529848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:54.529848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:54.529848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:54.529848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:54.529848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:54.529848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:54.529848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:54.529848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:54.529848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:54.530782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:52:54.530782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:52:54.530782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:54.530782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:54.530782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 258/510] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:54.531148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d0cc5-d267-4ccf-acfc-05b88cd3712d None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:54.545627 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:54.549032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9c2bf-30c7-491d-9ec9-d8b1dbd2b0c1 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:54.583459 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c097d-7d79-4494-bd51-7b9822f41c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:54.584244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c097d-7d79-4494-bd51-7b9822f41c12 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:54.584526 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c097d-7d79-4494-bd51-7b9822f41c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:54.584828 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c097d-7d79-4494-bd51-7b9822f41c12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:54.595339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c097d-7d79-4494-bd51-7b9822f41c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:54.596843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c097d-7d79-4494-bd51-7b9822f41c12 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:54.606270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:54.614854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-ef6c097d-7d79-4494-bd51-7b9822f41c12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:54.614854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:54.614854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:54.614854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:54.614854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:54.614854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:54.614854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:54.614854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:54.614854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:54.614854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:54.614854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:54.614854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:54.614854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:52:54.614854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:52:54.614854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:52:54.615888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:52:54.615888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:54.615888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:54.615888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:54.615888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:54.615888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:54.615888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:54.615888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:54.615888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:54.615888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:54.615888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:54.615888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:54.615888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:54.615888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:54.615888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:54.615888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:54.615888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:54.616983 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:52:54.616983 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:52:54.616983 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:54.616983 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:54.616983 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 253/511] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:54.616983 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c097d-7d79-4494-bd51-7b9822f41c12 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:54.635484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d0cc5-d267-4ccf-acfc-05b88cd3712d None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:54.669320 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d012a47-88c5-4b9a-9e12-f8efe4b2eea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:54.670031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d012a47-88c5-4b9a-9e12-f8efe4b2eea5 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:54.670321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d012a47-88c5-4b9a-9e12-f8efe4b2eea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:54.670600 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d012a47-88c5-4b9a-9e12-f8efe4b2eea5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:54.678170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d012a47-88c5-4b9a-9e12-f8efe4b2eea5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:54.679627 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d012a47-88c5-4b9a-9e12-f8efe4b2eea5 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:54.695011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-3d012a47-88c5-4b9a-9e12-f8efe4b2eea5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:52:54.695011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:54.695011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:54.695011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:54.695011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:54.695011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:54.695011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:54.695011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:54.695011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:54.695011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:54.695011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:54.695011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:54.695011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:52:54.695011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:52:54.695011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:52:54.695011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:54.696905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:54.698043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:54.698043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:54.698043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:52:54.698043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:52:54.698043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:52:54.698043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:52:54.698043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:52:54.698043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:52:54.698043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:52:54.698043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:54.698043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 259/512] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Jul 12 21:52:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:54.698043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d012a47-88c5-4b9a-9e12-f8efe4b2eea5 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:54.723700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c097d-7d79-4494-bd51-7b9822f41c12 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:54.757162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c2be1-47df-43bb-8e19-dbc8b9dfcbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:54.757817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c2be1-47df-43bb-8e19-dbc8b9dfcbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:54.758143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c2be1-47df-43bb-8e19-dbc8b9dfcbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:54.758422 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c2be1-47df-43bb-8e19-dbc8b9dfcbf1 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:54.759143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c2be1-47df-43bb-8e19-dbc8b9dfcbf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:54.759451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c2be1-47df-43bb-8e19-dbc8b9dfcbf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:54.760849 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c2be1-47df-43bb-8e19-dbc8b9dfcbf1 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:54.773158 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 254/513] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:54.774568 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c2be1-47df-43bb-8e19-dbc8b9dfcbf1 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:54.790733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:54.800479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d012a47-88c5-4b9a-9e12-f8efe4b2eea5 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:54.834131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa60e6-84f8-46c4-b35c-2a1dafc7cd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:54.834803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa60e6-84f8-46c4-b35c-2a1dafc7cd71 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:54.835087 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa60e6-84f8-46c4-b35c-2a1dafc7cd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:54.835371 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa60e6-84f8-46c4-b35c-2a1dafc7cd71 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:54.836743 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa60e6-84f8-46c4-b35c-2a1dafc7cd71 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:54.837050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa60e6-84f8-46c4-b35c-2a1dafc7cd71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:54.838612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa60e6-84f8-46c4-b35c-2a1dafc7cd71 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:54.840591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:54.876439 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c2be1-47df-43bb-8e19-dbc8b9dfcbf1 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:54.877328 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 260/514] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Jul 12 21:52:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:54.878215 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa60e6-84f8-46c4-b35c-2a1dafc7cd71 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:54.913652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d860a9-d3cc-4abd-b47f-b83d4983c6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:54.914312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d860a9-d3cc-4abd-b47f-b83d4983c6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:54.914706 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d860a9-d3cc-4abd-b47f-b83d4983c6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:54.915208 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d860a9-d3cc-4abd-b47f-b83d4983c6c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:54.915967 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d860a9-d3cc-4abd-b47f-b83d4983c6c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:54.916252 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d860a9-d3cc-4abd-b47f-b83d4983c6c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:54.917654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d860a9-d3cc-4abd-b47f-b83d4983c6c2 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:54.928745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 255/515] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:54.929498 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d860a9-d3cc-4abd-b47f-b83d4983c6c2 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:54.989222 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa60e6-84f8-46c4-b35c-2a1dafc7cd71 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:55.001926 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:55.024048 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062e7e3-2a8e-4d99-a4e0-460b19fbb214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:55.024730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062e7e3-2a8e-4d99-a4e0-460b19fbb214 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:55.025041 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062e7e3-2a8e-4d99-a4e0-460b19fbb214 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:55.025322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062e7e3-2a8e-4d99-a4e0-460b19fbb214 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:55.026047 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062e7e3-2a8e-4d99-a4e0-460b19fbb214 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:55.026361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062e7e3-2a8e-4d99-a4e0-460b19fbb214 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:55.027768 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062e7e3-2a8e-4d99-a4e0-460b19fbb214 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:55.033016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d860a9-d3cc-4abd-b47f-b83d4983c6c2 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:55.039126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 261/516] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:55.040244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062e7e3-2a8e-4d99-a4e0-460b19fbb214 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:55.072456 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba39ab-d3fa-4d0f-a58a-3cca32e98cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:55.073154 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba39ab-d3fa-4d0f-a58a-3cca32e98cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:55.073436 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba39ab-d3fa-4d0f-a58a-3cca32e98cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:55.073710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba39ab-d3fa-4d0f-a58a-3cca32e98cd7 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:55.074554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba39ab-d3fa-4d0f-a58a-3cca32e98cd7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:55.074862 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba39ab-d3fa-4d0f-a58a-3cca32e98cd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:55.076322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba39ab-d3fa-4d0f-a58a-3cca32e98cd7 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:55.094765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 256/517] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Jul 12 21:52:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:55.095597 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba39ab-d3fa-4d0f-a58a-3cca32e98cd7 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:55.101195 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:55.152263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062e7e3-2a8e-4d99-a4e0-460b19fbb214 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:55.185005 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f462d-11ae-4caa-bbc1-f8f24ad45c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:55.185716 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f462d-11ae-4caa-bbc1-f8f24ad45c4a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:55.186000 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f462d-11ae-4caa-bbc1-f8f24ad45c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:55.186275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f462d-11ae-4caa-bbc1-f8f24ad45c4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:55.193462 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f462d-11ae-4caa-bbc1-f8f24ad45c4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:55.194823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba39ab-d3fa-4d0f-a58a-3cca32e98cd7 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:55.194925 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f462d-11ae-4caa-bbc1-f8f24ad45c4a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:55.210917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-b03f462d-11ae-4caa-bbc1-f8f24ad45c4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:52:55.210917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:55.210917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:55.210917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:55.210917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:55.210917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:55.210917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:55.210917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:55.210917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:55.210917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:55.210917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:55.210917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:55.210917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:52:55.210917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:52:55.210917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:52:55.212398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:52:55.212398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:55.212398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:55.212398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:55.212398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:55.212398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:55.212398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:55.212398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:55.212398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:55.212398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:55.212398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:55.212398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:55.212398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:55.212398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:55.212398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:55.212398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:55.212398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:55.213260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:52:55.213260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:52:55.213260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:52:55.213260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:52:55.213260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:52:55.213260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:52:55.213260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:52:55.213260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:55.213260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 262/518] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:52:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:55.213745 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f462d-11ae-4caa-bbc1-f8f24ad45c4a None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:55.229527 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:55.235934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb3b86-e2f2-4f2c-89a0-2c067f8c4d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:55.236593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb3b86-e2f2-4f2c-89a0-2c067f8c4d43 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:55.236913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb3b86-e2f2-4f2c-89a0-2c067f8c4d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:55.237195 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb3b86-e2f2-4f2c-89a0-2c067f8c4d43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:55.306619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb3b86-e2f2-4f2c-89a0-2c067f8c4d43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:55.308091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb3b86-e2f2-4f2c-89a0-2c067f8c4d43 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:55.313275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f462d-11ae-4caa-bbc1-f8f24ad45c4a None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:55.327224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:55.353140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35c5a9-e157-44a8-8d7d-a753ee5c77b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:55.354188 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35c5a9-e157-44a8-8d7d-a753ee5c77b0 None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:55.354672 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35c5a9-e157-44a8-8d7d-a753ee5c77b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:55.355052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35c5a9-e157-44a8-8d7d-a753ee5c77b0 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:55.356257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35c5a9-e157-44a8-8d7d-a753ee5c77b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:55.358703 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35c5a9-e157-44a8-8d7d-a753ee5c77b0 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:55.384031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 263/519] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:52:55 2022] POST /identity/v3/users => generated 247 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:52:55.384962 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35c5a9-e157-44a8-8d7d-a753ee5c77b0 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:55.433904 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 257/520] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:55.434820 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb3b86-e2f2-4f2c-89a0-2c067f8c4d43 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:55.472221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:55.509389 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35c5a9-e157-44a8-8d7d-a753ee5c77b0 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:55.548011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d470b63-6739-4fc4-bd92-9ef9b9d7f5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:55.548844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d470b63-6739-4fc4-bd92-9ef9b9d7f5df None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:55.549514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d470b63-6739-4fc4-bd92-9ef9b9d7f5df None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:55.549514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d470b63-6739-4fc4-bd92-9ef9b9d7f5df None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:55.558365 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d470b63-6739-4fc4-bd92-9ef9b9d7f5df None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:55.559852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d470b63-6739-4fc4-bd92-9ef9b9d7f5df None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:55.563731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb3b86-e2f2-4f2c-89a0-2c067f8c4d43 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:55.577114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-0d470b63-6739-4fc4-bd92-9ef9b9d7f5df None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 21:52:55.577114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:55.577114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:55.577114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:55.577114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:55.577114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:55.577114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:55.577114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:55.577114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:55.577114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:55.577114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:55.577114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:55.577114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:52:55.577114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:52:55.577114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:52:55.577114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:55.578070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:55.578977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:55.578977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:55.578977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:52:55.578977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:52:55.578977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:52:55.578977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:52:55.578977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:52:55.578977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:52:55.578977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 21:52:55.578977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:55.578977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 264/521] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:52:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:55.579532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d470b63-6739-4fc4-bd92-9ef9b9d7f5df None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:55.609605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22699b-d9f8-4676-b86c-5089558dfbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:55.610368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22699b-d9f8-4676-b86c-5089558dfbcb None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:55.610666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22699b-d9f8-4676-b86c-5089558dfbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:55.610939 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22699b-d9f8-4676-b86c-5089558dfbcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:55.614717 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:55.687125 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22699b-d9f8-4676-b86c-5089558dfbcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:55.688809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22699b-d9f8-4676-b86c-5089558dfbcb None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:55.695782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d470b63-6739-4fc4-bd92-9ef9b9d7f5df None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:55.722543 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:55.731058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49abd0-bac4-462d-8df2-43689edf3e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:55.731729 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49abd0-bac4-462d-8df2-43689edf3e63 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:55.732017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49abd0-bac4-462d-8df2-43689edf3e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:55.732297 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49abd0-bac4-462d-8df2-43689edf3e63 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:55.733127 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49abd0-bac4-462d-8df2-43689edf3e63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:55.733447 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49abd0-bac4-462d-8df2-43689edf3e63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:55.734912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49abd0-bac4-462d-8df2-43689edf3e63 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:55.758663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 265/522] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:52:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:55.758663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49abd0-bac4-462d-8df2-43689edf3e63 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:55.810666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 258/523] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:55.810666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22699b-d9f8-4676-b86c-5089558dfbcb None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:55.853712 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:55.867086 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49abd0-bac4-462d-8df2-43689edf3e63 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:55.913861 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbf86d-bf25-43c6-bd22-057592dc09da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:55.915548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbf86d-bf25-43c6-bd22-057592dc09da None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:55.915835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbf86d-bf25-43c6-bd22-057592dc09da None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:55.916118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbf86d-bf25-43c6-bd22-057592dc09da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:55.919613 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22699b-d9f8-4676-b86c-5089558dfbcb None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:55.926920 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbf86d-bf25-43c6-bd22-057592dc09da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:55.928600 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbf86d-bf25-43c6-bd22-057592dc09da None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:55.948065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-bfbbf86d-bf25-43c6-bd22-057592dc09da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:55.948065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:55.948065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:55.948065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:55.948065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:55.948065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:55.948065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:55.948065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:55.948065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:55.948065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:55.948065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:55.948065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:55.948065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:52:55.948065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:52:55.948065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:52:55.949055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:52:55.949055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:55.949055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:55.949055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:55.949055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:55.949055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:55.949055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:55.949055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:55.949055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:55.949055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:55.949055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:55.949055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:55.949055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:55.949055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:55.949055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:55.949055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:55.949055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:55.950072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:52:55.950072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:52:55.950072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:55.950072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:55.950072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 266/524] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:55.950072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbf86d-bf25-43c6-bd22-057592dc09da None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:55.958570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:55.970950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9e096-4f91-4a26-9d97-772926b86d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:55.971636 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9e096-4f91-4a26-9d97-772926b86d39 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:55.971926 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9e096-4f91-4a26-9d97-772926b86d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:55.972208 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9e096-4f91-4a26-9d97-772926b86d39 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:55.973162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9e096-4f91-4a26-9d97-772926b86d39 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:55.973417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9e096-4f91-4a26-9d97-772926b86d39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:55.974850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9e096-4f91-4a26-9d97-772926b86d39 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:55.994507 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 259/525] 10.222.0.44 () {56 vars in 1139 bytes} [Tue Jul 12 21:52:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:55.995366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9e096-4f91-4a26-9d97-772926b86d39 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:56.087999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbf86d-bf25-43c6-bd22-057592dc09da None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:56.103234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9e096-4f91-4a26-9d97-772926b86d39 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:56.128906 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7e069-1707-46e5-adcc-df5da24a2923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:56.129640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7e069-1707-46e5-adcc-df5da24a2923 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:56.129946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7e069-1707-46e5-adcc-df5da24a2923 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:56.130217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7e069-1707-46e5-adcc-df5da24a2923 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:56.131112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7e069-1707-46e5-adcc-df5da24a2923 None admin] RBAC: Adding query filter params (role.id=f1e42ea5a1c440bba915b459d2744858, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=c53e1e801e5e40e794e4ccfff34ec988) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:56.131401 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7e069-1707-46e5-adcc-df5da24a2923 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:56.133667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7e069-1707-46e5-adcc-df5da24a2923 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:56.144753 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b0899-75a9-4693-8fa8-8eb029c02a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:56.144974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 267/526] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:55 2022] GET /identity/v3/role_assignments?user.id=c53e1e801e5e40e794e4ccfff34ec988&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=f1e42ea5a1c440bba915b459d2744858 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:56.145067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b0899-75a9-4693-8fa8-8eb029c02a91 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:56.145149 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7e069-1707-46e5-adcc-df5da24a2923 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:56.145250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b0899-75a9-4693-8fa8-8eb029c02a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:56.145250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b0899-75a9-4693-8fa8-8eb029c02a91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:56.158818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:56.159837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b0899-75a9-4693-8fa8-8eb029c02a91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:56.161542 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b0899-75a9-4693-8fa8-8eb029c02a91 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:56.184562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-ed8b0899-75a9-4693-8fa8-8eb029c02a91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:56.184562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:56.184562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:56.184562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:56.184562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:56.184562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:56.184562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:56.184562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:56.184562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:56.184562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:56.184562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:56.184562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:56.184562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:52:56.184562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:52:56.184562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:52:56.185710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:52:56.185710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:56.185710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:56.185710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:56.185710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:56.185710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:56.185710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:56.185710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:56.185710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:56.185710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:56.185710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:56.185710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:56.185710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:56.185710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:56.185710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:56.185710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:56.185710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:56.186784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:52:56.186784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:52:56.186784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:56.186784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:56.186784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 260/527] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:56.187206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b0899-75a9-4693-8fa8-8eb029c02a91 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:56.238929 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:56.282138 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7e069-1707-46e5-adcc-df5da24a2923 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:56.313520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b0899-75a9-4693-8fa8-8eb029c02a91 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:56.326769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1170f-898a-4bc5-ad13-c41669c8f1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:56.326769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1170f-898a-4bc5-ad13-c41669c8f1ff None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:56.326769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1170f-898a-4bc5-ad13-c41669c8f1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:56.326769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1170f-898a-4bc5-ad13-c41669c8f1ff None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:56.327162 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1170f-898a-4bc5-ad13-c41669c8f1ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:56.327162 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1170f-898a-4bc5-ad13-c41669c8f1ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:56.327492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1170f-898a-4bc5-ad13-c41669c8f1ff None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:56.348631 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 268/528] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:56.350099 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1170f-898a-4bc5-ad13-c41669c8f1ff None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:56.369149 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770aafd-c39f-4712-8983-ebe1ec371070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:56.370434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770aafd-c39f-4712-8983-ebe1ec371070 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:56.370972 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770aafd-c39f-4712-8983-ebe1ec371070 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:56.371673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770aafd-c39f-4712-8983-ebe1ec371070 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:56.423336 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:56.466434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1170f-898a-4bc5-ad13-c41669c8f1ff None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:56.475253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770aafd-c39f-4712-8983-ebe1ec371070 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:56.478019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770aafd-c39f-4712-8983-ebe1ec371070 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:56.513886 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2810e-e82b-4921-9b61-f9293550f76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:56.515841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2810e-e82b-4921-9b61-f9293550f76e None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:56.516359 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2810e-e82b-4921-9b61-f9293550f76e None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:56.516886 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2810e-e82b-4921-9b61-f9293550f76e None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:56.517859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2810e-e82b-4921-9b61-f9293550f76e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:56.518381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2810e-e82b-4921-9b61-f9293550f76e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:56.520642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2810e-e82b-4921-9b61-f9293550f76e None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:56.522312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:56.544325 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 269/529] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:56.545531 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2810e-e82b-4921-9b61-f9293550f76e None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:56.599819 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 261/530] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:52:56.600793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770aafd-c39f-4712-8983-ebe1ec371070 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:56.674217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2810e-e82b-4921-9b61-f9293550f76e None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:56.700009 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:56.712889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630726c-5556-4a05-9a7b-badd8697dbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:56.713530 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630726c-5556-4a05-9a7b-badd8697dbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:56.713818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630726c-5556-4a05-9a7b-badd8697dbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:56.714099 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630726c-5556-4a05-9a7b-badd8697dbf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:56.721560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630726c-5556-4a05-9a7b-badd8697dbf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:56.723217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630726c-5556-4a05-9a7b-badd8697dbf8 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:56.739249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-1630726c-5556-4a05-9a7b-badd8697dbf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:52:56.739249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:56.739249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:56.739249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:56.739249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:56.739249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:56.739249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:56.739249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:56.739249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:56.739249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:56.739249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:56.739249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:56.739249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:52:56.739249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:52:56.739249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:52:56.740168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:52:56.740168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:56.740168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:56.740168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:56.740168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:56.740168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:56.740168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:56.740168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:56.740168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:56.740168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:56.740168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:56.740168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:56.740168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:56.740168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:56.740168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:56.740168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:56.740168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:56.741050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:52:56.741050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:52:56.741050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:52:56.741050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:52:56.741050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:52:56.741050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:52:56.741050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:52:56.741050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:56.742615 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770aafd-c39f-4712-8983-ebe1ec371070 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:56.745879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 270/531] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:52:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:56.746160 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630726c-5556-4a05-9a7b-badd8697dbf8 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:56.788301 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04075209-ca59-4cb6-9fef-d5302b26f3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:56.789074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04075209-ca59-4cb6-9fef-d5302b26f3be None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:56.789897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04075209-ca59-4cb6-9fef-d5302b26f3be None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:56.790179 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04075209-ca59-4cb6-9fef-d5302b26f3be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:56.797436 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04075209-ca59-4cb6-9fef-d5302b26f3be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:56.798902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04075209-ca59-4cb6-9fef-d5302b26f3be None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:56.801530 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:56.814698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-04075209-ca59-4cb6-9fef-d5302b26f3be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:52:56.814698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:56.814698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:56.814698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:56.814698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:56.814698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:56.814698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:56.814698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:56.814698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:56.814698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:56.814698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:56.814698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:56.814698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:52:56.814698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:52:56.814698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:52:56.815591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:52:56.815591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:56.815591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:56.815591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:56.815591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:56.815591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:56.815591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:56.815591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:56.815591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:56.815591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:56.815591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:56.815591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:56.815591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:56.815591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:56.815591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:56.815591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:56.815591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:56.816738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:52:56.816738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:52:56.816738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:52:56.816738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:52:56.816738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:52:56.816738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:52:56.816738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:52:56.816738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:56.816738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 262/532] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:52:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:56.817687 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04075209-ca59-4cb6-9fef-d5302b26f3be None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:56.876942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630726c-5556-4a05-9a7b-badd8697dbf8 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:56.926185 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d69ad-8351-416b-9898-a5208d6b1881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:56.927609 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04075209-ca59-4cb6-9fef-d5302b26f3be None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:56.928348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d69ad-8351-416b-9898-a5208d6b1881 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:56.928860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d69ad-8351-416b-9898-a5208d6b1881 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:56.929611 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d69ad-8351-416b-9898-a5208d6b1881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:56.946362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d69ad-8351-416b-9898-a5208d6b1881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:56.948576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d69ad-8351-416b-9898-a5208d6b1881 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:56.970326 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643fede-583b-445e-b9bb-fdf1f28ece59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:56.971131 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643fede-583b-445e-b9bb-fdf1f28ece59 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:56.971488 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643fede-583b-445e-b9bb-fdf1f28ece59 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:56.971840 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643fede-583b-445e-b9bb-fdf1f28ece59 None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:56.972727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643fede-583b-445e-b9bb-fdf1f28ece59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:56.973116 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643fede-583b-445e-b9bb-fdf1f28ece59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:56.973950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-2f6d69ad-8351-416b-9898-a5208d6b1881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:56.973950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:56.973950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:56.973950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:56.973950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:56.973950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:56.973950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:56.973950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:56.973950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:56.973950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:56.973950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:56.973950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:56.973950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:52:56.973950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:52:56.973950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:52:56.974760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:52:56.974760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:56.974760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:56.974760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:56.974760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:56.974760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:56.974760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:56.974760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:56.974760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:56.974760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:56.974760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:56.974760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:56.974760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:56.974760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:56.974760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:56.974760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:56.974760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:56.975586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:52:56.975586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:52:56.975586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:56.975586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:56.975811 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643fede-583b-445e-b9bb-fdf1f28ece59 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:56.978275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 271/533] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:56.979332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d69ad-8351-416b-9898-a5208d6b1881 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:57.000806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 263/534] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:52:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:57.001754 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643fede-583b-445e-b9bb-fdf1f28ece59 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:57.030803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:57.089032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:57.103354 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d69ad-8351-416b-9898-a5208d6b1881 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:57.136724 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643fede-583b-445e-b9bb-fdf1f28ece59 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:57.139811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db5ef5-e0dd-48d5-b3ad-44b84002a716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:57.140577 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db5ef5-e0dd-48d5-b3ad-44b84002a716 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:57.140981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db5ef5-e0dd-48d5-b3ad-44b84002a716 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:57.141350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db5ef5-e0dd-48d5-b3ad-44b84002a716 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:57.142241 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db5ef5-e0dd-48d5-b3ad-44b84002a716 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:57.142672 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db5ef5-e0dd-48d5-b3ad-44b84002a716 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:57.144206 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db5ef5-e0dd-48d5-b3ad-44b84002a716 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:57.161071 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 272/535] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:52:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:57.162091 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db5ef5-e0dd-48d5-b3ad-44b84002a716 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:57.173752 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b5088-80e8-4210-a87e-bdaf9b72c0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:57.174427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b5088-80e8-4210-a87e-bdaf9b72c0af None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:57.174712 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b5088-80e8-4210-a87e-bdaf9b72c0af None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:57.174993 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b5088-80e8-4210-a87e-bdaf9b72c0af None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:57.175726 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43b5088-80e8-4210-a87e-bdaf9b72c0af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:57.176035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43b5088-80e8-4210-a87e-bdaf9b72c0af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:57.177470 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43b5088-80e8-4210-a87e-bdaf9b72c0af None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:57.193069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 264/536] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:57.194639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b5088-80e8-4210-a87e-bdaf9b72c0af None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:57.276078 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:57.282971 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db5ef5-e0dd-48d5-b3ad-44b84002a716 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:57.305909 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b5088-80e8-4210-a87e-bdaf9b72c0af None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:57.321600 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871d535-0e87-4b32-bd93-61e25cb218b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:57.322714 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871d535-0e87-4b32-bd93-61e25cb218b6 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:57.322996 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871d535-0e87-4b32-bd93-61e25cb218b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:57.323540 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871d535-0e87-4b32-bd93-61e25cb218b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:57.324385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871d535-0e87-4b32-bd93-61e25cb218b6 None admin] RBAC: Adding query filter params (role.id=f1e42ea5a1c440bba915b459d2744858, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=c5c9ed6f9fd542069cf914003b636c1c) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:57.324872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871d535-0e87-4b32-bd93-61e25cb218b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:57.326701 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871d535-0e87-4b32-bd93-61e25cb218b6 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:57.328798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:57.334498 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 273/537] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:57 2022] GET /identity/v3/role_assignments?user.id=c5c9ed6f9fd542069cf914003b636c1c&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=f1e42ea5a1c440bba915b459d2744858 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:57.335903 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871d535-0e87-4b32-bd93-61e25cb218b6 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:57.381670 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196ed38-99db-47b1-9564-519632cadeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:57.382322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196ed38-99db-47b1-9564-519632cadeab None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:57.382608 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196ed38-99db-47b1-9564-519632cadeab None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:57.382912 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196ed38-99db-47b1-9564-519632cadeab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:57.383803 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196ed38-99db-47b1-9564-519632cadeab None admin] RBAC: Adding query filter params (role.id=f1e42ea5a1c440bba915b459d2744858, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=24915625354f418eb7bc4d5ada85fdb0) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:57.384120 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196ed38-99db-47b1-9564-519632cadeab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:57.385635 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196ed38-99db-47b1-9564-519632cadeab None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:57.403186 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 265/538] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:57 2022] GET /identity/v3/role_assignments?user.id=24915625354f418eb7bc4d5ada85fdb0&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=f1e42ea5a1c440bba915b459d2744858 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:57.483584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871d535-0e87-4b32-bd93-61e25cb218b6 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:57.516667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54c1a3-7968-4087-b8f6-2d6361eae148 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:57.517109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54c1a3-7968-4087-b8f6-2d6361eae148 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:57.517307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54c1a3-7968-4087-b8f6-2d6361eae148 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:57.522021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 266/539] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:57.525541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a2f43-df1a-489a-910c-b43fd00a98b3 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:57.525974 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a2f43-df1a-489a-910c-b43fd00a98b3 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:57.526356 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a2f43-df1a-489a-910c-b43fd00a98b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:57.529404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32ee49-a427-4aa5-b7cb-018bbea529d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:57.530382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32ee49-a427-4aa5-b7cb-018bbea529d3 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:57.530664 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32ee49-a427-4aa5-b7cb-018bbea529d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:57.530938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32ee49-a427-4aa5-b7cb-018bbea529d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:57.542075 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32ee49-a427-4aa5-b7cb-018bbea529d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:57.543585 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32ee49-a427-4aa5-b7cb-018bbea529d3 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:57.550976 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-472a2f43-df1a-489a-910c-b43fd00a98b3 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:57.552218 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-472a2f43-df1a-489a-910c-b43fd00a98b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:57.561831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-4d32ee49-a427-4aa5-b7cb-018bbea529d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:52:57.561831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:57.561831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:57.561831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:57.561831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:57.561831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:57.561831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:57.561831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:57.561831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:57.561831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:57.561831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:57.561831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:57.561831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:52:57.561831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:52:57.561831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:52:57.563642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:52:57.563642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:57.563642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:57.563642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:57.563642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:57.563642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:57.563642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:57.563642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:57.563642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:57.563642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:57.563642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:57.563642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:57.563642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:57.563642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:57.563642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:57.563642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:57.563642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:57.565265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:52:57.565265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:52:57.565265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:52:57.565265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:52:57.565265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:52:57.565265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:52:57.565265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:52:57.565265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:57.565265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 274/540] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:52:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:57.571347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32ee49-a427-4aa5-b7cb-018bbea529d3 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:57.602379 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:57.701150 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32ee49-a427-4aa5-b7cb-018bbea529d3 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:57.712271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 267/541] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:52:57.738786 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af15e1-44c6-49c7-8851-0c493b05c908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:57.739452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af15e1-44c6-49c7-8851-0c493b05c908 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:57.739783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af15e1-44c6-49c7-8851-0c493b05c908 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:57.740063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af15e1-44c6-49c7-8851-0c493b05c908 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:57.740968 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af15e1-44c6-49c7-8851-0c493b05c908 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:57.741279 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af15e1-44c6-49c7-8851-0c493b05c908 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:57.742894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af15e1-44c6-49c7-8851-0c493b05c908 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:57.747200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a2f43-df1a-489a-910c-b43fd00a98b3 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:57.756053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 275/542] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:52:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:52:57.762642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af15e1-44c6-49c7-8851-0c493b05c908 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:57.855433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:57.872568 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af15e1-44c6-49c7-8851-0c493b05c908 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:57.948553 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d49e65-4c05-4dcf-a592-71be4e85aa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:57.949250 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d49e65-4c05-4dcf-a592-71be4e85aa94 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:57.949548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d49e65-4c05-4dcf-a592-71be4e85aa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:57.949830 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d49e65-4c05-4dcf-a592-71be4e85aa94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:57.950784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d49e65-4c05-4dcf-a592-71be4e85aa94 None admin] RBAC: Adding query filter params (role.id=f1e42ea5a1c440bba915b459d2744858, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=53b891a734a64b5983564e207de0359d) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:57.951108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d49e65-4c05-4dcf-a592-71be4e85aa94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:57.952751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d49e65-4c05-4dcf-a592-71be4e85aa94 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:57.977149 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a2f43-df1a-489a-910c-b43fd00a98b3 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:57.980348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 276/543] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:52:57 2022] GET /identity/v3/role_assignments?user.id=53b891a734a64b5983564e207de0359d&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=f1e42ea5a1c440bba915b459d2744858 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:58.014090 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:58.041051 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016880d-348a-4a35-b31d-e2a7bee2afcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:58.041886 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016880d-348a-4a35-b31d-e2a7bee2afcb None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:58.042168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016880d-348a-4a35-b31d-e2a7bee2afcb None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:58.042445 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016880d-348a-4a35-b31d-e2a7bee2afcb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:58.061604 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016880d-348a-4a35-b31d-e2a7bee2afcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:58.069861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-4016880d-348a-4a35-b31d-e2a7bee2afcb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:52:58.069861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:58.069861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:58.069861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:58.069861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:58.069861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:58.069861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:58.069861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:58.069861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:58.069861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:58.069861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:58.069861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:58.069861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:58.069861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:58.069861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:58.069861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:58.071234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:58.071234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:58.071234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:58.071234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:58.071234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:58.071234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:58.071234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:58.071234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:58.071234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:58.071234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:58.071234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:58.071234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:58.071234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:58.071234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:58.071234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:58.071234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:58.072041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:58.072041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:58.072041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:58.072041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:58.072041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:58.072041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:58.072041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:58.072041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:58.072041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:58.072041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:58.072041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:52:58.072041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:58.072041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 268/544] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:52:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:58.077245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d49e65-4c05-4dcf-a592-71be4e85aa94 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:58.235792 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d49e65-4c05-4dcf-a592-71be4e85aa94 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:58.282870 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9284da1-8c86-4945-84f8-9f576088ce3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:58.283553 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9284da1-8c86-4945-84f8-9f576088ce3f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:58.283840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9284da1-8c86-4945-84f8-9f576088ce3f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:58.284124 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9284da1-8c86-4945-84f8-9f576088ce3f None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:58.284918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9284da1-8c86-4945-84f8-9f576088ce3f None admin] RBAC: Adding query filter params (name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:58.285231 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9284da1-8c86-4945-84f8-9f576088ce3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:58.286622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9284da1-8c86-4945-84f8-9f576088ce3f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:58.299609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 277/545] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:52:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:58.305316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016880d-348a-4a35-b31d-e2a7bee2afcb None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:58.405674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeecedf-cda4-427e-91fa-401687baa87b None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:58.405955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeecedf-cda4-427e-91fa-401687baa87b None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:58.406226 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeecedf-cda4-427e-91fa-401687baa87b None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:58.407851 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 278/546] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:58.412881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab42a3-f3da-4ea4-bdfc-289596ca7a31 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:58.413174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab42a3-f3da-4ea4-bdfc-289596ca7a31 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:58.413459 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab42a3-f3da-4ea4-bdfc-289596ca7a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:58.441263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:58.450152 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-1eab42a3-f3da-4ea4-bdfc-289596ca7a31 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:52:58.451396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-1eab42a3-f3da-4ea4-bdfc-289596ca7a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:52:58.477503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016880d-348a-4a35-b31d-e2a7bee2afcb None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:58.541052 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf66b0-3903-40d5-9344-a8c5ba74c1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:58.541852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf66b0-3903-40d5-9344-a8c5ba74c1af None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:58.542138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf66b0-3903-40d5-9344-a8c5ba74c1af None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:58.542417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf66b0-3903-40d5-9344-a8c5ba74c1af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:58.561334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf66b0-3903-40d5-9344-a8c5ba74c1af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:58.562826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf66b0-3903-40d5-9344-a8c5ba74c1af None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:58.583056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 279/547] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:52:58.592074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-4fcf66b0-3903-40d5-9344-a8c5ba74c1af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:58.592074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:58.592074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:58.592074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:58.592074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:58.592074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:58.592074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:58.592074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:58.592074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:58.592074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:58.592074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:58.592074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:58.592074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:52:58.592074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:52:58.592074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:52:58.594108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:52:58.594108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:58.594108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:58.594108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:58.594108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:58.594108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:58.594108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:58.594108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:58.594108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:58.594108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:58.594108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:58.594108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:58.594108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:58.594108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:58.594108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:58.594108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:58.594108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:58.596126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:52:58.596126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:52:58.596126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:58.596126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:58.596126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 269/548] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:58.605321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab42a3-f3da-4ea4-bdfc-289596ca7a31 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:58.631733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf66b0-3903-40d5-9344-a8c5ba74c1af None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:58.671846 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:58.798282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab42a3-f3da-4ea4-bdfc-289596ca7a31 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:58.812440 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf66b0-3903-40d5-9344-a8c5ba74c1af None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:58.868120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba89922-528d-4394-90f7-bcb0bb0399ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:58.869256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba89922-528d-4394-90f7-bcb0bb0399ed None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:58.870197 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba89922-528d-4394-90f7-bcb0bb0399ed None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:58.871018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba89922-528d-4394-90f7-bcb0bb0399ed None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:58.872372 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba89922-528d-4394-90f7-bcb0bb0399ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:58.873323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba89922-528d-4394-90f7-bcb0bb0399ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:58.875371 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba89922-528d-4394-90f7-bcb0bb0399ed None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:58.878666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7b0cd-43fa-483c-9625-c977cec100e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:58.879692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7b0cd-43fa-483c-9625-c977cec100e3 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:58.880143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7b0cd-43fa-483c-9625-c977cec100e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:58.880590 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7b0cd-43fa-483c-9625-c977cec100e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:58.884043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:58.903144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 280/549] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:58.903902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7b0cd-43fa-483c-9625-c977cec100e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:52:58.911955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-3ce7b0cd-43fa-483c-9625-c977cec100e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:52:58.911955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:58.911955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:58.911955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:58.911955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:58.911955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:58.911955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:58.911955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:58.911955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:58.911955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:58.911955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:58.911955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:58.911955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:52:58.911955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:52:58.911955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:52:58.911955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:52:58.913560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:52:58.913560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:52:58.913560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:52:58.913560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:52:58.913560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:58.913560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:58.913560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:58.913560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:58.913560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:58.913560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:58.913560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:58.913560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:58.913560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:58.913560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:58.913560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:58.913560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:58.915548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:58.915548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:58.915548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:58.915548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:58.915548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:52:58.915548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:52:58.915548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:52:58.915548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:52:58.915548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:52:58.915548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:52:58.915548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:52:58.915548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:58.917355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba89922-528d-4394-90f7-bcb0bb0399ed None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:58.918256 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 270/550] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:52:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:52:58.925563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7b0cd-43fa-483c-9625-c977cec100e3 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:59.125146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:59.143083 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba89922-528d-4394-90f7-bcb0bb0399ed None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:59.160402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7b0cd-43fa-483c-9625-c977cec100e3 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:59.198884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bef541-4ceb-4aa8-bdd5-8982406d4656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:59.199689 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bef541-4ceb-4aa8-bdd5-8982406d4656 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:59.199976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bef541-4ceb-4aa8-bdd5-8982406d4656 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:59.200257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bef541-4ceb-4aa8-bdd5-8982406d4656 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:59.215359 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bef541-4ceb-4aa8-bdd5-8982406d4656 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:59.216880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bef541-4ceb-4aa8-bdd5-8982406d4656 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:59.243260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-d4bef541-4ceb-4aa8-bdd5-8982406d4656 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 21:52:59.243260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:59.243260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:59.243260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:59.243260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:59.243260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:59.243260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:59.243260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:59.243260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:59.243260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:59.243260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:59.243260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:59.243260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:52:59.243260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:52:59.243260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:52:59.243260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:59.244220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:59.245944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:59.245944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:59.245944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:52:59.245944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:52:59.245944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:52:59.245944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:52:59.245944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:52:59.245944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:52:59.245944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 21:52:59.245944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:52:59.245944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 281/551] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:52:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:59.246518 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8084d9-dd74-48f9-971b-f9e8001ee201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:59.246518 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8084d9-dd74-48f9-971b-f9e8001ee201 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:59.246518 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8084d9-dd74-48f9-971b-f9e8001ee201 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:59.246518 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8084d9-dd74-48f9-971b-f9e8001ee201 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:59.248668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8084d9-dd74-48f9-971b-f9e8001ee201 None admin] RBAC: Adding query filter params (name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:59.248668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8084d9-dd74-48f9-971b-f9e8001ee201 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:59.249209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8084d9-dd74-48f9-971b-f9e8001ee201 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:59.256007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bef541-4ceb-4aa8-bdd5-8982406d4656 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:59.266301 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 271/552] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:52:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:59.280875 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8084d9-dd74-48f9-971b-f9e8001ee201 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:59.399586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8084d9-dd74-48f9-971b-f9e8001ee201 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:59.437094 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:59.452132 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ce9e1-6892-428e-9bdf-72cabdb22c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:59.452874 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ce9e1-6892-428e-9bdf-72cabdb22c30 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:59.453363 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ce9e1-6892-428e-9bdf-72cabdb22c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:59.453656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ce9e1-6892-428e-9bdf-72cabdb22c30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:59.467753 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ce9e1-6892-428e-9bdf-72cabdb22c30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:59.469443 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ce9e1-6892-428e-9bdf-72cabdb22c30 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:59.490237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:59.493784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-937ce9e1-6892-428e-9bdf-72cabdb22c30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:59.493784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:59.493784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:59.493784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:59.493784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:59.493784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:59.493784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:59.493784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:59.493784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:59.493784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:59.493784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:59.493784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:59.493784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:52:59.493784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:52:59.493784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:52:59.494672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:52:59.494672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:59.494672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:59.494672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:59.494672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:59.494672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:59.494672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:59.494672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:59.494672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:59.494672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:59.494672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:59.494672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:59.494672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:59.494672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:59.494672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:59.494672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:59.494672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:59.495628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:52:59.495628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:52:59.495628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:52:59.495628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:59.495628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 272/553] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:52:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:52:59.502035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ce9e1-6892-428e-9bdf-72cabdb22c30 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:59.574672 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bef541-4ceb-4aa8-bdd5-8982406d4656 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:59.615352 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ce9e1-6892-428e-9bdf-72cabdb22c30 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:59.655113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3274bc2-53f6-45c9-b83a-c648e8b3c97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:59.655705 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3274bc2-53f6-45c9-b83a-c648e8b3c97a None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:59.655995 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3274bc2-53f6-45c9-b83a-c648e8b3c97a None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:59.656276 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3274bc2-53f6-45c9-b83a-c648e8b3c97a None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:59.657173 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3274bc2-53f6-45c9-b83a-c648e8b3c97a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:59.657495 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3274bc2-53f6-45c9-b83a-c648e8b3c97a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:59.659050 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3274bc2-53f6-45c9-b83a-c648e8b3c97a None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:59.677038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 273/554] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:59.678792 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3274bc2-53f6-45c9-b83a-c648e8b3c97a None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:59.690399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4915430-b8b6-426c-a4a1-18c69d41839f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:59.691089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4915430-b8b6-426c-a4a1-18c69d41839f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:59.691624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4915430-b8b6-426c-a4a1-18c69d41839f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:59.691913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4915430-b8b6-426c-a4a1-18c69d41839f None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:59.692786 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4915430-b8b6-426c-a4a1-18c69d41839f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:52:59.693102 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4915430-b8b6-426c-a4a1-18c69d41839f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:59.694548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4915430-b8b6-426c-a4a1-18c69d41839f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:59.732774 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 282/555] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Jul 12 21:52:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 477 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:52:59.737870 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4915430-b8b6-426c-a4a1-18c69d41839f None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:59.806975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:59.863403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4915430-b8b6-426c-a4a1-18c69d41839f None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:59.867938 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3274bc2-53f6-45c9-b83a-c648e8b3c97a None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:52:59.878711 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:52:59.906687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaa1dd-c83f-46f1-885c-d3c50dee1952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:59.907257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaa1dd-c83f-46f1-885c-d3c50dee1952 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:59.907545 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaa1dd-c83f-46f1-885c-d3c50dee1952 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:59.907850 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaa1dd-c83f-46f1-885c-d3c50dee1952 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:59.909622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d138d-911c-418a-b50a-312033b0efc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:52:59.910582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d138d-911c-418a-b50a-312033b0efc8 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:59.911109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d138d-911c-418a-b50a-312033b0efc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:59.911538 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d138d-911c-418a-b50a-312033b0efc8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:59.919872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d138d-911c-418a-b50a-312033b0efc8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:59.921443 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d138d-911c-418a-b50a-312033b0efc8 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:52:59.937404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-bc2d138d-911c-418a-b50a-312033b0efc8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 21:52:59.937404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:52:59.937404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:52:59.937404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:52:59.937404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:52:59.937404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:52:59.937404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:52:59.937404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:52:59.937404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:52:59.937404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:52:59.937404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:52:59.937404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:52:59.937404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:52:59.937404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:52:59.937404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:52:59.937404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:52:59.938850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:52:59.939788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:52:59.939788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:52:59.939788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:52:59.939788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:52:59.939788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:52:59.939788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:52:59.939788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:52:59.939788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:52:59.939788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 21:52:59.939788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:52:59.939788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 274/556] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:52:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:52:59.940385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03171c-8dda-43be-a6e5-1c7d074d650b None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:59.940385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03171c-8dda-43be-a6e5-1c7d074d650b None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:59.940515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03171c-8dda-43be-a6e5-1c7d074d650b None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:59.942281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 275/557] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:59.943188 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218f8f2-a15b-4e31-b402-7bf7c6494aa9 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:52:59.943762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218f8f2-a15b-4e31-b402-7bf7c6494aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:52:59.944102 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218f8f2-a15b-4e31-b402-7bf7c6494aa9 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:52:59.949258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 276/558] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:52:59.949258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218f8f2-a15b-4e31-b402-7bf7c6494aa9 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:52:59.998444 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaa1dd-c83f-46f1-885c-d3c50dee1952 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:52:59.999940 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaa1dd-c83f-46f1-885c-d3c50dee1952 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:00.065539 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218f8f2-a15b-4e31-b402-7bf7c6494aa9 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:00.088815 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:00.104728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1d6d6-147b-4fb5-8065-ecde06f74a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:00.105425 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1d6d6-147b-4fb5-8065-ecde06f74a74 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:00.105713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1d6d6-147b-4fb5-8065-ecde06f74a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:00.106000 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1d6d6-147b-4fb5-8065-ecde06f74a74 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:00.106828 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1d6d6-147b-4fb5-8065-ecde06f74a74 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:00.107140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1d6d6-147b-4fb5-8065-ecde06f74a74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:00.108573 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1d6d6-147b-4fb5-8065-ecde06f74a74 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:00.111862 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 283/559] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:52:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 21:53:00.113211 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9b6fe-836a-401d-9dea-8d70828579bc None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:00.113505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9b6fe-836a-401d-9dea-8d70828579bc None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:00.113788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9b6fe-836a-401d-9dea-8d70828579bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:00.130675 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-86a9b6fe-836a-401d-9dea-8d70828579bc None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:00.131940 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-86a9b6fe-836a-401d-9dea-8d70828579bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:00.132130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 277/560] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Jul 12 21:52:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:00.134722 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e88a9-d7fd-4330-a3ff-c848b91629e7 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:00.135145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e88a9-d7fd-4330-a3ff-c848b91629e7 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:00.135582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e88a9-d7fd-4330-a3ff-c848b91629e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:00.147357 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:00.152775 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-8c8e88a9-d7fd-4330-a3ff-c848b91629e7 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:00.154100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8c8e88a9-d7fd-4330-a3ff-c848b91629e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:00.216826 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 284/561] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:53:00.216826 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9b6fe-836a-401d-9dea-8d70828579bc None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:00.229817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 278/562] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:53:00.230944 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e88a9-d7fd-4330-a3ff-c848b91629e7 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:00.336858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9b6fe-836a-401d-9dea-8d70828579bc None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:00.343088 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e88a9-d7fd-4330-a3ff-c848b91629e7 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:00.354910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:00.375469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b0d9a-991f-4b9d-804f-2a25db63e41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:00.376266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b0d9a-991f-4b9d-804f-2a25db63e41a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:00.376552 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b0d9a-991f-4b9d-804f-2a25db63e41a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:00.377055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b0d9a-991f-4b9d-804f-2a25db63e41a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:00.385403 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9177112-02d3-4877-a5d9-fe09398094d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:00.386519 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9177112-02d3-4877-a5d9-fe09398094d4 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:00.386519 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9177112-02d3-4877-a5d9-fe09398094d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:00.386741 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9177112-02d3-4877-a5d9-fe09398094d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:00.388735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b0d9a-991f-4b9d-804f-2a25db63e41a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:00.390217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b0d9a-991f-4b9d-804f-2a25db63e41a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:00.393133 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:00.397356 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9177112-02d3-4877-a5d9-fe09398094d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:00.399593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9177112-02d3-4877-a5d9-fe09398094d4 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:00.410632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-7c1b0d9a-991f-4b9d-804f-2a25db63e41a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:00.410632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:00.410632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:00.410632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:00.410632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:00.410632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:00.410632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:00.410632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:00.410632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:00.410632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:00.410632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:00.410632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:00.410632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:00.410632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:00.410632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:00.411489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:00.411489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:00.411489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:00.411489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:00.411489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:00.411489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:00.411489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:00.411489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:00.411489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:00.411489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:00.411489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:00.411489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:00.411489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:00.411489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:00.411489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:00.411489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:00.411489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:00.412425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:00.412425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:00.412425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:00.412425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:00.414984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 285/563] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:00.416216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b0d9a-991f-4b9d-804f-2a25db63e41a None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:00.422361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-c9177112-02d3-4877-a5d9-fe09398094d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:00.422361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:00.422361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:00.422361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:00.422361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:00.422361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:00.422361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:00.422361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:00.422361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:00.422361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:00.422361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:00.422361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:00.422361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:00.422361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:00.422361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:00.423235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:00.423235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:00.423235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:00.423235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:00.423235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:00.423235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:00.423235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:00.423235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:00.423235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:00.423235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:00.423235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:00.423235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:00.423235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:00.423235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:00.423235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:00.423235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:00.423235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:00.424147 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:00.424147 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:00.424147 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:00.424147 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:00.424147 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 279/564] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:00.424853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9177112-02d3-4877-a5d9-fe09398094d4 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:00.525533 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b0d9a-991f-4b9d-804f-2a25db63e41a None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:00.535259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9177112-02d3-4877-a5d9-fe09398094d4 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:00.559179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06629783-cf4a-4d2a-9113-f7bbfbf67bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:00.559967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06629783-cf4a-4d2a-9113-f7bbfbf67bdd None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:00.560251 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06629783-cf4a-4d2a-9113-f7bbfbf67bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:00.560527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06629783-cf4a-4d2a-9113-f7bbfbf67bdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:00.569291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c3cf0-1e38-4a5f-81c0-a059beeb06db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:00.570084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c3cf0-1e38-4a5f-81c0-a059beeb06db None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:00.571207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c3cf0-1e38-4a5f-81c0-a059beeb06db None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:00.571359 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06629783-cf4a-4d2a-9113-f7bbfbf67bdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:00.571518 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c3cf0-1e38-4a5f-81c0-a059beeb06db None admin] PATH_INFO: `/v3/roles/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:00.579175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-06629783-cf4a-4d2a-9113-f7bbfbf67bdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:00.579175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:00.579175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:00.579175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:00.579175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:00.579175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:00.579175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:00.579175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:00.579175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:00.579175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:00.579175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:00.579175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:00.579175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:00.579175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:00.579175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:00.579175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:00.580282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:00.580282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:00.580282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:00.580282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:00.580282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:00.580282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:00.580282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:00.580282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:00.580282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:00.580282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:00.580282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:00.580282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:00.580282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:00.580282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:00.580282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:00.580282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:00.586265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:00.586265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:00.586265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:00.586265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:00.586265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:00.586265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:00.586265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:00.586265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:00.586265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:00.586265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:00.586265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:00.586265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:00.586265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 286/565] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:53:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:00.586265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06629783-cf4a-4d2a-9113-f7bbfbf67bdd None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:00.587022 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c3cf0-1e38-4a5f-81c0-a059beeb06db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:00.589625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-190c3cf0-1e38-4a5f-81c0-a059beeb06db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:00.589625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:00.589625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:00.589625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:00.589625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:00.589625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:00.589625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:00.589625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:00.589625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:00.589625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:00.589625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:00.589625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:00.589625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:00.589625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:00.589625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:00.589625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:00.591406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:00.591406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:00.591406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:00.591406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:00.591406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:00.591406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:00.591406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:00.591406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:00.591406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:00.591406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:00.591406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:00.591406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:00.591406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:00.591406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:00.591406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:00.591406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:00.593033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:00.593033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:00.593033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:00.593033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:00.593033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:00.593033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:00.593033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:00.593033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:00.593033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:00.593033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:00.593033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:00.593033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:00.593033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 280/566] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:53:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:00.593033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c3cf0-1e38-4a5f-81c0-a059beeb06db None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:00.615303 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:00.642653 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:00.688180 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06629783-cf4a-4d2a-9113-f7bbfbf67bdd None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:00.696931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c3cf0-1e38-4a5f-81c0-a059beeb06db None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:00.722547 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41a68e-a529-4613-b43d-1a67bb91d86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:00.723478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41a68e-a529-4613-b43d-1a67bb91d86b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:00.723768 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41a68e-a529-4613-b43d-1a67bb91d86b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:00.724080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41a68e-a529-4613-b43d-1a67bb91d86b None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:00.724801 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41a68e-a529-4613-b43d-1a67bb91d86b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:00.725113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41a68e-a529-4613-b43d-1a67bb91d86b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:00.726487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41a68e-a529-4613-b43d-1a67bb91d86b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:00.730120 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f72cd-6993-4544-9b7b-b55e6648a7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:00.730784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f72cd-6993-4544-9b7b-b55e6648a7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:00.731070 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f72cd-6993-4544-9b7b-b55e6648a7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:00.731369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f72cd-6993-4544-9b7b-b55e6648a7e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:00.732099 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f72cd-6993-4544-9b7b-b55e6648a7e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:00.732409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f72cd-6993-4544-9b7b-b55e6648a7e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:00.733812 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f72cd-6993-4544-9b7b-b55e6648a7e9 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:00.738980 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 287/567] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:00.740348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9044a9-3fe0-47ab-9603-037aaab3c7b4 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:00.741159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9044a9-3fe0-47ab-9603-037aaab3c7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:00.741452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9044a9-3fe0-47ab-9603-037aaab3c7b4 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:00.743818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 288/568] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:00.750754 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 281/569] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:00.750754 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f72cd-6993-4544-9b7b-b55e6648a7e9 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:00.750961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9044a9-3fe0-47ab-9603-037aaab3c7b4 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:00.835037 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:00.854580 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9044a9-3fe0-47ab-9603-037aaab3c7b4 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:00.855634 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f72cd-6993-4544-9b7b-b55e6648a7e9 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:00.870767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:00.890083 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb344069-47bb-4a7b-94a6-ccaa4c40ec5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:00.890748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb344069-47bb-4a7b-94a6-ccaa4c40ec5f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:00.891030 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb344069-47bb-4a7b-94a6-ccaa4c40ec5f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:00.891303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb344069-47bb-4a7b-94a6-ccaa4c40ec5f None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:00.892054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb344069-47bb-4a7b-94a6-ccaa4c40ec5f None admin] RBAC: Adding query filter params (name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:00.892143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5ed78-5236-4e8a-bb1f-95d78f2cc910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:00.892366 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb344069-47bb-4a7b-94a6-ccaa4c40ec5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:00.892755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5ed78-5236-4e8a-bb1f-95d78f2cc910 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:00.893042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5ed78-5236-4e8a-bb1f-95d78f2cc910 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:00.893321 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5ed78-5236-4e8a-bb1f-95d78f2cc910 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:00.893781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb344069-47bb-4a7b-94a6-ccaa4c40ec5f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:00.894068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5ed78-5236-4e8a-bb1f-95d78f2cc910 None admin] RBAC: Adding query filter params (name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:00.894380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5ed78-5236-4e8a-bb1f-95d78f2cc910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:00.895744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5ed78-5236-4e8a-bb1f-95d78f2cc910 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:00.902685 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 289/570] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:53:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:00.903492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb344069-47bb-4a7b-94a6-ccaa4c40ec5f None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:00.904737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 282/571] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:53:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:00.905965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ca9b6-146d-4ef7-b6b9-6b417e6391e6 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:00.906272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ca9b6-146d-4ef7-b6b9-6b417e6391e6 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:00.906531 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ca9b6-146d-4ef7-b6b9-6b417e6391e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:00.922559 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-575ca9b6-146d-4ef7-b6b9-6b417e6391e6 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:00.923757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-575ca9b6-146d-4ef7-b6b9-6b417e6391e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:00.995259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 283/572] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:53:00.996461 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ca9b6-146d-4ef7-b6b9-6b417e6391e6 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:01.021741 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb344069-47bb-4a7b-94a6-ccaa4c40ec5f None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:01.057411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa9767-4e01-470f-8306-92fc4d12bbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:01.058226 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa9767-4e01-470f-8306-92fc4d12bbe3 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:01.058511 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa9767-4e01-470f-8306-92fc4d12bbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:01.058789 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa9767-4e01-470f-8306-92fc4d12bbe3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:01.065688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa9767-4e01-470f-8306-92fc4d12bbe3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:01.067424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa9767-4e01-470f-8306-92fc4d12bbe3 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:01.072242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:01.088182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-62aa9767-4e01-470f-8306-92fc4d12bbe3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:01.088182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:01.088182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:01.088182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:01.088182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:01.088182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:01.088182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:01.088182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:01.088182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:01.088182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:01.088182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:01.088182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:01.088182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:01.088182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:01.088182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:01.089769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:01.089769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:01.089769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:01.089769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:01.089769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:01.089769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:01.089769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:01.089769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:01.089769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:01.089769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:01.089769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:01.089769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:01.089769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:01.089769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:01.089769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:01.089769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:01.089769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:01.091912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:01.091912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:01.091912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:01.091912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:01.091912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:01.091912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:01.091912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:01.091912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:01.091912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 290/573] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:01.091912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa9767-4e01-470f-8306-92fc4d12bbe3 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:01.105800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ca9b6-146d-4ef7-b6b9-6b417e6391e6 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:01.136475 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:01.142107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0648469-251c-408c-9beb-4b2d5c3f012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:01.142908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0648469-251c-408c-9beb-4b2d5c3f012c None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:01.143190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0648469-251c-408c-9beb-4b2d5c3f012c None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:01.143466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0648469-251c-408c-9beb-4b2d5c3f012c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:01.150281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0648469-251c-408c-9beb-4b2d5c3f012c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:01.151759 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0648469-251c-408c-9beb-4b2d5c3f012c None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:01.165945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-b0648469-251c-408c-9beb-4b2d5c3f012c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:01.165945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:01.165945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:01.165945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:01.165945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:01.165945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:01.165945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:01.165945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:01.165945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:01.165945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:01.165945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:01.165945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:01.165945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:01.165945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:01.165945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:01.166930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:01.166930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:01.166930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:01.166930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:01.166930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:01.166930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:01.166930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:01.166930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:01.166930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:01.166930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:01.166930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:01.166930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:01.166930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:01.166930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:01.166930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:01.166930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:01.166930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:01.167800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:01.167800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:01.167800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:01.167800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:01.167800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:01.167800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:01.167800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:01.167800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:01.167800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 284/574] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:01.168283 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0648469-251c-408c-9beb-4b2d5c3f012c None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:01.204167 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa9767-4e01-470f-8306-92fc4d12bbe3 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:01.238195 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e3ee6-c979-4b46-b673-3dcfdaed8791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:01.238983 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e3ee6-c979-4b46-b673-3dcfdaed8791 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:01.239264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e3ee6-c979-4b46-b673-3dcfdaed8791 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:01.239539 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e3ee6-c979-4b46-b673-3dcfdaed8791 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:01.250392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e3ee6-c979-4b46-b673-3dcfdaed8791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:01.251849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e3ee6-c979-4b46-b673-3dcfdaed8791 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:01.267617 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0648469-251c-408c-9beb-4b2d5c3f012c None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:01.269015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-bb0e3ee6-c979-4b46-b673-3dcfdaed8791 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:01.269015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:01.269015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:01.269015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:01.269015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:01.269015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:01.269015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:01.269015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:01.269015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:01.269015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:01.269015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:01.269015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:01.269015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:01.269015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:01.269015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:01.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:01.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:01.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:01.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:01.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:01.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:01.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:01.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:01.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:01.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:01.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:01.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:01.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:01.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:01.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:01.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:01.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:01.270967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:01.270967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:01.270967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:01.270967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:01.270967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 291/575] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:01.270967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e3ee6-c979-4b46-b673-3dcfdaed8791 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:01.300972 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:01.308037 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02b216-0b36-4544-ba22-1d2a2b86b99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:01.308874 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02b216-0b36-4544-ba22-1d2a2b86b99c None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:01.309610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02b216-0b36-4544-ba22-1d2a2b86b99c None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:01.309610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02b216-0b36-4544-ba22-1d2a2b86b99c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:01.321560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b02b216-0b36-4544-ba22-1d2a2b86b99c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:01.323066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b02b216-0b36-4544-ba22-1d2a2b86b99c None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:01.342067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-4b02b216-0b36-4544-ba22-1d2a2b86b99c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:01.342067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:01.342067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:01.342067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:01.342067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:01.342067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:01.342067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:01.342067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:01.342067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:01.342067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:01.342067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:01.342067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:01.342067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:01.342067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:01.342067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:01.343653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:01.343653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:01.343653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:01.343653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:01.343653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:01.343653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:01.343653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:01.343653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:01.343653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:01.343653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:01.343653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:01.343653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:01.343653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:01.343653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:01.343653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:01.343653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:01.343653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:01.344678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:01.344678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:01.344678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:01.344678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:01.344678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 285/576] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:01.344678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02b216-0b36-4544-ba22-1d2a2b86b99c None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:01.375537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:01.387519 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e3ee6-c979-4b46-b673-3dcfdaed8791 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:01.431167 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49307ac-33bb-4db1-ac24-8fdbdf8b16be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:01.431995 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49307ac-33bb-4db1-ac24-8fdbdf8b16be None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:01.432281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49307ac-33bb-4db1-ac24-8fdbdf8b16be None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:01.432553 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49307ac-33bb-4db1-ac24-8fdbdf8b16be None admin] PATH_INFO: `/v3/roles/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:01.443209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49307ac-33bb-4db1-ac24-8fdbdf8b16be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:01.451270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-f49307ac-33bb-4db1-ac24-8fdbdf8b16be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:01.451270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:01.451270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:01.451270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:01.451270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:01.451270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:01.451270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:01.451270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:01.451270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:01.451270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:01.451270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:01.451270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:01.451270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:01.451270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:01.451270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:01.451270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:01.452262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:01.452262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:01.452262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:01.452262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:01.452262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:01.452262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:01.452262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:01.452262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:01.452262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:01.452262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:01.452262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:01.452262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:01.452262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:01.452262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:01.452262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:01.452262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:01.453118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:01.453118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:01.453118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:01.453118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:01.453118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:01.453118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:01.453118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:01.453118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:01.453118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:01.453118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:01.453118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:01.453118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:01.453118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 292/577] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:53:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:01.453792 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49307ac-33bb-4db1-ac24-8fdbdf8b16be None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:01.457782 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02b216-0b36-4544-ba22-1d2a2b86b99c None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:01.494499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dfdcb-33b9-414f-894b-f232395c339d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:01.495170 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dfdcb-33b9-414f-894b-f232395c339d None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:01.495456 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dfdcb-33b9-414f-894b-f232395c339d None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:01.495733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dfdcb-33b9-414f-894b-f232395c339d None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:01.496935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95dfdcb-33b9-414f-894b-f232395c339d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:01.497251 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95dfdcb-33b9-414f-894b-f232395c339d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:01.499004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95dfdcb-33b9-414f-894b-f232395c339d None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:01.511383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 286/578] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:01.512205 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dfdcb-33b9-414f-894b-f232395c339d None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:01.554940 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49307ac-33bb-4db1-ac24-8fdbdf8b16be None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:01.560582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:01.589429 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa9a43-348c-41e3-ba72-461eece5fed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:01.590083 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa9a43-348c-41e3-ba72-461eece5fed9 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:01.590372 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa9a43-348c-41e3-ba72-461eece5fed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:01.590643 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa9a43-348c-41e3-ba72-461eece5fed9 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:01.591424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa9a43-348c-41e3-ba72-461eece5fed9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:01.591730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa9a43-348c-41e3-ba72-461eece5fed9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:01.593251 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa9a43-348c-41e3-ba72-461eece5fed9 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:01.605959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 293/579] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:01.606771 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa9a43-348c-41e3-ba72-461eece5fed9 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:01.617094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dfdcb-33b9-414f-894b-f232395c339d None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:01.620364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:01.652020 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341254b-c2e3-4b9e-889a-95750f2738e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:01.652690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341254b-c2e3-4b9e-889a-95750f2738e5 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:01.653004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341254b-c2e3-4b9e-889a-95750f2738e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:01.653283 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341254b-c2e3-4b9e-889a-95750f2738e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:01.654008 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341254b-c2e3-4b9e-889a-95750f2738e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:01.654314 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341254b-c2e3-4b9e-889a-95750f2738e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:01.655704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341254b-c2e3-4b9e-889a-95750f2738e5 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:01.667913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 287/580] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:01.668796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341254b-c2e3-4b9e-889a-95750f2738e5 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:01.715675 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa9a43-348c-41e3-ba72-461eece5fed9 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:01.751231 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71868d-85cb-4e56-bb8e-e924cf625791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:01.752146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71868d-85cb-4e56-bb8e-e924cf625791 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:01.752420 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71868d-85cb-4e56-bb8e-e924cf625791 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:01.752692 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71868d-85cb-4e56-bb8e-e924cf625791 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:01.753447 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71868d-85cb-4e56-bb8e-e924cf625791 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:01.753744 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71868d-85cb-4e56-bb8e-e924cf625791 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:01.755111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71868d-85cb-4e56-bb8e-e924cf625791 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:01.766377 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 294/581] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:01.767189 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71868d-85cb-4e56-bb8e-e924cf625791 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:01.776446 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341254b-c2e3-4b9e-889a-95750f2738e5 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:01.778844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:01.812583 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248393d-9c85-4731-9681-d592314b9ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:01.813227 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248393d-9c85-4731-9681-d592314b9ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:01.813518 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248393d-9c85-4731-9681-d592314b9ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:01.813796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248393d-9c85-4731-9681-d592314b9ec4 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:01.814541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248393d-9c85-4731-9681-d592314b9ec4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:01.814852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248393d-9c85-4731-9681-d592314b9ec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:01.816223 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248393d-9c85-4731-9681-d592314b9ec4 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:01.824834 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 288/582] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:53:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:01.825708 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248393d-9c85-4731-9681-d592314b9ec4 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:01.839240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:01.874553 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71868d-85cb-4e56-bb8e-e924cf625791 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:01.909468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681bec2-578b-4ca5-9f30-994cc38ccdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:01.910378 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681bec2-578b-4ca5-9f30-994cc38ccdef None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:01.910659 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681bec2-578b-4ca5-9f30-994cc38ccdef None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:01.910932 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681bec2-578b-4ca5-9f30-994cc38ccdef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:01.911759 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681bec2-578b-4ca5-9f30-994cc38ccdef None admin] RBAC: Adding query filter params (role.id=f1e42ea5a1c440bba915b459d2744858, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=837bb5ac18554dfdbbcc776689639494) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:01.912072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681bec2-578b-4ca5-9f30-994cc38ccdef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:01.913573 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681bec2-578b-4ca5-9f30-994cc38ccdef None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:01.919207 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 295/583] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:53:01 2022] GET /identity/v3/role_assignments?user.id=837bb5ac18554dfdbbcc776689639494&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=f1e42ea5a1c440bba915b459d2744858 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:01.919830 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681bec2-578b-4ca5-9f30-994cc38ccdef None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:01.941032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248393d-9c85-4731-9681-d592314b9ec4 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:01.990459 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758028a5-128d-403e-b5aa-14291df571eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:01.991501 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758028a5-128d-403e-b5aa-14291df571eb None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:01.991866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758028a5-128d-403e-b5aa-14291df571eb None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:01.992246 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758028a5-128d-403e-b5aa-14291df571eb None admin] PATH_INFO: `/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/c53e1e801e5e40e794e4ccfff34ec988/roles/f1e42ea5a1c440bba915b459d2744858` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:02.009089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758028a5-128d-403e-b5aa-14291df571eb None admin] RBAC: Authorizing `identity:create_grant(project_id=459e4743ff3043f69408ca2095be3bd0, user_id=c53e1e801e5e40e794e4ccfff34ec988, role_id=f1e42ea5a1c440bba915b459d2744858)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:02.010580 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758028a5-128d-403e-b5aa-14291df571eb None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:02.035988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 289/584] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:53:01 2022] PUT /identity/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/c53e1e801e5e40e794e4ccfff34ec988/roles/f1e42ea5a1c440bba915b459d2744858 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:53:02.035988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758028a5-128d-403e-b5aa-14291df571eb None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:02.049980 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681bec2-578b-4ca5-9f30-994cc38ccdef None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:02.064897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:02.096648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39113467-106a-441c-a862-27701f484fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:02.097543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39113467-106a-441c-a862-27701f484fdf None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:02.097829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39113467-106a-441c-a862-27701f484fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:02.098107 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39113467-106a-441c-a862-27701f484fdf None admin] PATH_INFO: `/v3/users/glance` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:02.109679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39113467-106a-441c-a862-27701f484fdf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:02.111481 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39113467-106a-441c-a862-27701f484fdf None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:02.121110 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:02.135310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-39113467-106a-441c-a862-27701f484fdf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 21:53:02.135310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:02.135310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:02.135310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:02.135310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:02.135310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:02.135310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:02.135310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:02.135310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:02.135310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:02.135310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:02.135310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:02.135310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:02.135310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:02.135310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:02.135310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:02.136270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:02.137281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:02.137281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:02.137281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:02.137281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:02.137281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:02.137281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:02.137281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:02.137281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:02.137281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 21:53:02.137281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:02.139764 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 296/585] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:53:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:02.140802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39113467-106a-441c-a862-27701f484fdf None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:02.174490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758028a5-128d-403e-b5aa-14291df571eb None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:02.215012 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625185f7-b114-406b-b134-2f431cb5ae64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:02.215818 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625185f7-b114-406b-b134-2f431cb5ae64 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:02.216094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625185f7-b114-406b-b134-2f431cb5ae64 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:02.216467 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625185f7-b114-406b-b134-2f431cb5ae64 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:02.226009 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625185f7-b114-406b-b134-2f431cb5ae64 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:02.228465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625185f7-b114-406b-b134-2f431cb5ae64 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:02.248080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-625185f7-b114-406b-b134-2f431cb5ae64 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:53:02.248080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:02.248080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:02.248080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:02.248080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:02.248080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:02.248080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:02.248080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:02.248080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:02.248080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:02.248080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:02.248080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:02.248080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:02.248080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:02.248080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:02.248080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:02.249123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:02.250098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:02.250098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:02.250098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:02.250098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:02.250098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:02.250098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:02.250098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:02.250098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:02.250098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:53:02.250098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:02.250632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 290/586] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Jul 12 21:53:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:02.251244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625185f7-b114-406b-b134-2f431cb5ae64 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:02.293373 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39113467-106a-441c-a862-27701f484fdf None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:02.330581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970d058-36ed-4fd9-8655-e7b1eeaa05c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:02.331477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970d058-36ed-4fd9-8655-e7b1eeaa05c8 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:02.331859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970d058-36ed-4fd9-8655-e7b1eeaa05c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:02.332143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970d058-36ed-4fd9-8655-e7b1eeaa05c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:02.343832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970d058-36ed-4fd9-8655-e7b1eeaa05c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:02.345451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970d058-36ed-4fd9-8655-e7b1eeaa05c8 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:02.356843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:02.366936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-e970d058-36ed-4fd9-8655-e7b1eeaa05c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:02.366936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:02.366936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:02.366936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:02.366936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:02.366936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:02.366936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:02.366936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:02.366936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:02.366936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:02.366936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:02.366936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:02.366936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:02.366936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:02.366936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:02.368765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:02.368765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:02.368765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:02.368765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:02.368765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:02.368765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:02.368765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:02.368765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:02.368765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:02.368765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:02.368765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:02.368765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:02.368765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:02.368765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:02.368765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:02.368765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:02.368765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:02.369844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:02.369844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:02.369844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:02.369844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:02.376147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 297/587] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:02.377493 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970d058-36ed-4fd9-8655-e7b1eeaa05c8 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:02.384109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625185f7-b114-406b-b134-2f431cb5ae64 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:02.430455 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fe420-b263-467c-b4a4-31d7e132a112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:02.431533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fe420-b263-467c-b4a4-31d7e132a112 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:02.431823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fe420-b263-467c-b4a4-31d7e132a112 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:02.432104 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fe420-b263-467c-b4a4-31d7e132a112 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:02.433404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:02.434458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9fe420-b263-467c-b4a4-31d7e132a112 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:02.435089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9fe420-b263-467c-b4a4-31d7e132a112 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:02.437549 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9fe420-b263-467c-b4a4-31d7e132a112 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:02.464787 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 291/588] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Jul 12 21:53:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:02.465736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fe420-b263-467c-b4a4-31d7e132a112 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:02.496482 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970d058-36ed-4fd9-8655-e7b1eeaa05c8 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:02.534477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af06e86-fd7b-4b3b-8191-0f7b1fb711ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:02.535543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af06e86-fd7b-4b3b-8191-0f7b1fb711ba None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:02.536021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af06e86-fd7b-4b3b-8191-0f7b1fb711ba None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:02.536492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af06e86-fd7b-4b3b-8191-0f7b1fb711ba None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:02.537565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af06e86-fd7b-4b3b-8191-0f7b1fb711ba None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:02.538095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af06e86-fd7b-4b3b-8191-0f7b1fb711ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:02.539943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af06e86-fd7b-4b3b-8191-0f7b1fb711ba None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:02.561198 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 298/589] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Jul 12 21:53:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:02.562579 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af06e86-fd7b-4b3b-8191-0f7b1fb711ba None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:02.603576 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fe420-b263-467c-b4a4-31d7e132a112 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:02.654105 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e7d1d-2726-4fd4-bdeb-27a90cfb0a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:02.655028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e7d1d-2726-4fd4-bdeb-27a90cfb0a14 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:02.655319 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e7d1d-2726-4fd4-bdeb-27a90cfb0a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:02.655609 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e7d1d-2726-4fd4-bdeb-27a90cfb0a14 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:02.656895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e7d1d-2726-4fd4-bdeb-27a90cfb0a14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:02.657220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e7d1d-2726-4fd4-bdeb-27a90cfb0a14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:02.658905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e7d1d-2726-4fd4-bdeb-27a90cfb0a14 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:02.660520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:02.677638 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 292/590] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:02.682623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e7d1d-2726-4fd4-bdeb-27a90cfb0a14 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:02.684390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af06e86-fd7b-4b3b-8191-0f7b1fb711ba None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:02.688069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:02.727614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0906e94-acbc-484a-9617-ffc8f9947fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:02.728842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0906e94-acbc-484a-9617-ffc8f9947fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:02.729476 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0906e94-acbc-484a-9617-ffc8f9947fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:02.730183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0906e94-acbc-484a-9617-ffc8f9947fc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:02.744670 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0906e94-acbc-484a-9617-ffc8f9947fc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:02.746645 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0906e94-acbc-484a-9617-ffc8f9947fc6 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:02.767791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-b0906e94-acbc-484a-9617-ffc8f9947fc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:02.767791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:02.767791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:02.767791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:02.767791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:02.767791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:02.767791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:02.767791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:02.767791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:02.767791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:02.767791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:02.767791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:02.767791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:02.767791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:02.767791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:02.768735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:02.768735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:02.768735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:02.768735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:02.768735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:02.768735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:02.768735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:02.768735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:02.768735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:02.768735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:02.768735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:02.768735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:02.768735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:02.768735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:02.768735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:02.768735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:02.768735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:02.769617 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:02.769617 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:02.769617 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:02.769617 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:02.771836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 299/591] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:02.772876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0906e94-acbc-484a-9617-ffc8f9947fc6 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:02.823974 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e7d1d-2726-4fd4-bdeb-27a90cfb0a14 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:02.860256 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451229d-e9fe-4031-b049-7bbaf5037409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:02.860849 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451229d-e9fe-4031-b049-7bbaf5037409 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:02.861142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451229d-e9fe-4031-b049-7bbaf5037409 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:02.861422 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451229d-e9fe-4031-b049-7bbaf5037409 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:02.873479 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451229d-e9fe-4031-b049-7bbaf5037409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:02.874946 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451229d-e9fe-4031-b049-7bbaf5037409 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:02.887759 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0906e94-acbc-484a-9617-ffc8f9947fc6 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:02.893244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-f451229d-e9fe-4031-b049-7bbaf5037409 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:02.893244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:02.893244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:02.893244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:02.893244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:02.893244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:02.893244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:02.893244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:02.893244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:02.893244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:02.893244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:02.893244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:02.893244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:02.893244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:02.893244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:02.894115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:02.894115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:02.894115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:02.894115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:02.894115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:02.894115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:02.894115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:02.894115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:02.894115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:02.894115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:02.894115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:02.894115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:02.894115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:02.894115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:02.894115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:02.894115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:02.894115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:02.894987 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:02.894987 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:02.894987 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:02.894987 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:02.894987 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 293/592] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:02.895712 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451229d-e9fe-4031-b049-7bbaf5037409 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:02.928912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60882aaf-f082-410f-b580-fd25dbe4f9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:02.930413 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60882aaf-f082-410f-b580-fd25dbe4f9aa None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:02.930927 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60882aaf-f082-410f-b580-fd25dbe4f9aa None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:02.931414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60882aaf-f082-410f-b580-fd25dbe4f9aa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:02.939477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60882aaf-f082-410f-b580-fd25dbe4f9aa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:02.941201 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60882aaf-f082-410f-b580-fd25dbe4f9aa None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:02.946745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:02.958463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-60882aaf-f082-410f-b580-fd25dbe4f9aa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 21:53:02.958463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:02.958463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:02.958463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:02.958463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:02.958463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:02.958463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:02.958463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:02.958463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:02.958463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:02.958463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:02.958463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:02.958463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:02.958463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:02.958463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:02.958463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:02.959407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:02.960344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:02.960344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:02.960344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:02.960344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:02.960344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:02.960344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:02.960344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:02.960344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:02.960344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 21:53:02.960344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:02.967750 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 300/593] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:53:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:02.968942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60882aaf-f082-410f-b580-fd25dbe4f9aa None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:02.989163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:03.006432 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451229d-e9fe-4031-b049-7bbaf5037409 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:03.042616 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce438cc2-260a-4849-be11-e12b652c1d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:03.043652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce438cc2-260a-4849-be11-e12b652c1d49 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:03.043652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce438cc2-260a-4849-be11-e12b652c1d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:03.043853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce438cc2-260a-4849-be11-e12b652c1d49 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:03.044582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce438cc2-260a-4849-be11-e12b652c1d49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:03.044916 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce438cc2-260a-4849-be11-e12b652c1d49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:03.046308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce438cc2-260a-4849-be11-e12b652c1d49 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:03.060902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 294/594] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:03.061827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce438cc2-260a-4849-be11-e12b652c1d49 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:03.088372 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60882aaf-f082-410f-b580-fd25dbe4f9aa None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:03.130620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e5b49-af1d-4b93-a0f2-90877027f397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:03.132292 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e5b49-af1d-4b93-a0f2-90877027f397 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:03.133520 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e5b49-af1d-4b93-a0f2-90877027f397 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:03.134019 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e5b49-af1d-4b93-a0f2-90877027f397 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:03.134945 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e5b49-af1d-4b93-a0f2-90877027f397 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:03.135446 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e5b49-af1d-4b93-a0f2-90877027f397 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:03.137227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e5b49-af1d-4b93-a0f2-90877027f397 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:03.150386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 301/595] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:03.151585 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e5b49-af1d-4b93-a0f2-90877027f397 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:03.194379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce438cc2-260a-4849-be11-e12b652c1d49 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:03.202959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:03.232985 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b655573-9c02-4cd3-b5f3-8796716cb8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:03.233751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b655573-9c02-4cd3-b5f3-8796716cb8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:03.234010 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b655573-9c02-4cd3-b5f3-8796716cb8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:03.234299 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b655573-9c02-4cd3-b5f3-8796716cb8c4 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:03.235149 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b655573-9c02-4cd3-b5f3-8796716cb8c4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:03.235465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b655573-9c02-4cd3-b5f3-8796716cb8c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:03.236735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:03.237084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b655573-9c02-4cd3-b5f3-8796716cb8c4 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:03.256529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 295/596] 10.222.0.44 () {56 vars in 1139 bytes} [Tue Jul 12 21:53:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:03.258333 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b655573-9c02-4cd3-b5f3-8796716cb8c4 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:03.264369 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e5b49-af1d-4b93-a0f2-90877027f397 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:03.301232 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21252cf8-c9f9-49ee-8f89-321efedb10e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:03.302842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21252cf8-c9f9-49ee-8f89-321efedb10e3 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:03.303351 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21252cf8-c9f9-49ee-8f89-321efedb10e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:03.303823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21252cf8-c9f9-49ee-8f89-321efedb10e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:03.311957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21252cf8-c9f9-49ee-8f89-321efedb10e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:03.313695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21252cf8-c9f9-49ee-8f89-321efedb10e3 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:03.329056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-21252cf8-c9f9-49ee-8f89-321efedb10e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:03.329056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:03.329056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:03.329056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:03.329056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:03.329056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:03.329056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:03.329056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:03.329056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:03.329056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:03.329056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:03.329056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:03.329056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:03.329056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:03.329056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:03.329956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:03.329956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:03.329956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:03.329956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:03.329956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:03.329956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:03.329956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:03.329956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:03.329956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:03.329956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:03.329956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:03.329956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:03.329956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:03.329956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:03.329956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:03.329956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:03.329956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:03.330823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:03.330823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:03.330823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:03.330823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:03.330823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:03.330823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:03.330823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:03.330823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:03.333332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 302/597] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:03.334337 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21252cf8-c9f9-49ee-8f89-321efedb10e3 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:03.376118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b655573-9c02-4cd3-b5f3-8796716cb8c4 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:03.412762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd0df8-3761-48cd-a1b3-1d69a76718cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:03.413564 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd0df8-3761-48cd-a1b3-1d69a76718cd None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:03.413846 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd0df8-3761-48cd-a1b3-1d69a76718cd None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:03.415024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd0df8-3761-48cd-a1b3-1d69a76718cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:03.423150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd0df8-3761-48cd-a1b3-1d69a76718cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:03.424681 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd0df8-3761-48cd-a1b3-1d69a76718cd None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:03.439888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-69dd0df8-3761-48cd-a1b3-1d69a76718cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:03.439888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:03.439888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:03.439888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:03.439888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:03.439888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:03.439888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:03.439888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:03.439888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:03.439888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:03.439888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:03.439888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:03.439888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:03.439888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:03.439888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:03.441160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:03.441160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:03.441160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:03.441160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:03.441160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:03.441160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:03.441160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:03.441160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:03.441160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:03.441160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:03.441160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:03.441160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:03.441160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:03.441160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:03.441160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:03.441160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:03.441160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:03.442291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:03.442291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:03.442291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:03.442291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:03.442291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:03.442291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:03.442291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:03.442291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:03.443784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21252cf8-c9f9-49ee-8f89-321efedb10e3 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:03.447979 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 296/598] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:03.448127 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd0df8-3761-48cd-a1b3-1d69a76718cd None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:03.476188 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:03.482296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be48c8-8c27-40d5-95e8-56a315a1523b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:03.483396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be48c8-8c27-40d5-95e8-56a315a1523b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:03.483875 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be48c8-8c27-40d5-95e8-56a315a1523b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:03.484391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be48c8-8c27-40d5-95e8-56a315a1523b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:03.487473 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:03.497053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be48c8-8c27-40d5-95e8-56a315a1523b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:03.498676 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be48c8-8c27-40d5-95e8-56a315a1523b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:03.517313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-23be48c8-8c27-40d5-95e8-56a315a1523b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:03.517313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:03.517313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:03.517313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:03.517313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:03.517313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:03.517313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:03.517313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:03.517313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:03.517313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:03.517313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:03.517313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:03.517313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:03.517313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:03.517313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:03.518266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:03.518266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:03.518266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:03.518266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:03.518266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:03.518266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:03.518266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:03.518266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:03.518266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:03.518266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:03.518266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:03.518266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:03.518266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:03.518266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:03.518266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:03.518266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:03.518266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:03.519131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:03.519131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:03.519131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:03.519131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:03.519131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 303/599] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:03.525009 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be48c8-8c27-40d5-95e8-56a315a1523b None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:03.563759 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd0df8-3761-48cd-a1b3-1d69a76718cd None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:03.602480 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5b460-0b52-4fe3-9349-e095487e3d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:03.603585 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5b460-0b52-4fe3-9349-e095487e3d4e None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:03.604164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5b460-0b52-4fe3-9349-e095487e3d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:03.604616 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5b460-0b52-4fe3-9349-e095487e3d4e None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:03.605613 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5b460-0b52-4fe3-9349-e095487e3d4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:03.606108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5b460-0b52-4fe3-9349-e095487e3d4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:03.607884 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5b460-0b52-4fe3-9349-e095487e3d4e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:03.627041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 297/600] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:03.628515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5b460-0b52-4fe3-9349-e095487e3d4e None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:03.630774 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be48c8-8c27-40d5-95e8-56a315a1523b None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:03.668791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d84eb-c985-4bf1-a925-e466af827c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:03.670297 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d84eb-c985-4bf1-a925-e466af827c53 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:03.670297 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d84eb-c985-4bf1-a925-e466af827c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:03.670297 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d84eb-c985-4bf1-a925-e466af827c53 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:03.670978 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d84eb-c985-4bf1-a925-e466af827c53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:03.671352 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d84eb-c985-4bf1-a925-e466af827c53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:03.672882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d84eb-c985-4bf1-a925-e466af827c53 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:03.689900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 304/601] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:03.690789 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d84eb-c985-4bf1-a925-e466af827c53 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:03.728286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:03.743318 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:03.749208 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5b460-0b52-4fe3-9349-e095487e3d4e None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:03.788109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd512e-570b-4044-8a74-658d2f2d1284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:03.789349 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd512e-570b-4044-8a74-658d2f2d1284 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:03.789816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd512e-570b-4044-8a74-658d2f2d1284 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:03.790312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd512e-570b-4044-8a74-658d2f2d1284 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:03.791444 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd512e-570b-4044-8a74-658d2f2d1284 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:03.791978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd512e-570b-4044-8a74-658d2f2d1284 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:03.793792 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd512e-570b-4044-8a74-658d2f2d1284 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:03.797447 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d84eb-c985-4bf1-a925-e466af827c53 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:03.807880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 298/602] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:03.809089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd512e-570b-4044-8a74-658d2f2d1284 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:03.838572 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c2351-8dc8-4026-9124-ab0509b24231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:03.839464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c2351-8dc8-4026-9124-ab0509b24231 None admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:03.839744 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c2351-8dc8-4026-9124-ab0509b24231 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:03.840021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c2351-8dc8-4026-9124-ab0509b24231 None admin] PATH_INFO: `/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/c5c9ed6f9fd542069cf914003b636c1c/roles/f1e42ea5a1c440bba915b459d2744858` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:03.856638 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c2351-8dc8-4026-9124-ab0509b24231 None admin] RBAC: Authorizing `identity:create_grant(project_id=459e4743ff3043f69408ca2095be3bd0, user_id=c5c9ed6f9fd542069cf914003b636c1c, role_id=f1e42ea5a1c440bba915b459d2744858)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:03.858258 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c2351-8dc8-4026-9124-ab0509b24231 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:03.873680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 305/603] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:53:03 2022] PUT /identity/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/c5c9ed6f9fd542069cf914003b636c1c/roles/f1e42ea5a1c440bba915b459d2744858 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:53:03.875582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c2351-8dc8-4026-9124-ab0509b24231 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:03.948093 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd512e-570b-4044-8a74-658d2f2d1284 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:04.026606 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da6a84-3cc3-4916-843b-ba56ff32ad94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:04.027527 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da6a84-3cc3-4916-843b-ba56ff32ad94 None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:04.027815 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da6a84-3cc3-4916-843b-ba56ff32ad94 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:04.028133 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da6a84-3cc3-4916-843b-ba56ff32ad94 None admin] PATH_INFO: `/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/24915625354f418eb7bc4d5ada85fdb0/roles/f1e42ea5a1c440bba915b459d2744858` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:04.066927 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:04.071803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c2351-8dc8-4026-9124-ab0509b24231 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:04.078788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da6a84-3cc3-4916-843b-ba56ff32ad94 None admin] RBAC: Authorizing `identity:create_grant(project_id=459e4743ff3043f69408ca2095be3bd0, user_id=24915625354f418eb7bc4d5ada85fdb0, role_id=f1e42ea5a1c440bba915b459d2744858)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:04.080278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da6a84-3cc3-4916-843b-ba56ff32ad94 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:04.082936 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:04.107800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 299/604] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:53:03 2022] PUT /identity/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/24915625354f418eb7bc4d5ada85fdb0/roles/f1e42ea5a1c440bba915b459d2744858 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:53:04.109080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5a4d2-8a48-4e05-9c95-139cd8b42477 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:04.109366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5a4d2-8a48-4e05-9c95-139cd8b42477 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:04.109646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5a4d2-8a48-4e05-9c95-139cd8b42477 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:04.111234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 300/605] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:04.119272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98df4a8-f5d9-407e-b17b-34516b8b7b00 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:04.119612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98df4a8-f5d9-407e-b17b-34516b8b7b00 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:04.119819 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98df4a8-f5d9-407e-b17b-34516b8b7b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:04.129130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2da3d7-0798-4c68-845b-e0bfd4a70184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:04.129947 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2da3d7-0798-4c68-845b-e0bfd4a70184 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:04.130233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2da3d7-0798-4c68-845b-e0bfd4a70184 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:04.130508 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2da3d7-0798-4c68-845b-e0bfd4a70184 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:04.145712 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2da3d7-0798-4c68-845b-e0bfd4a70184 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:04.147521 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2da3d7-0798-4c68-845b-e0bfd4a70184 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:04.152165 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-c98df4a8-f5d9-407e-b17b-34516b8b7b00 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:04.153452 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c98df4a8-f5d9-407e-b17b-34516b8b7b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:04.172392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-7f2da3d7-0798-4c68-845b-e0bfd4a70184 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:04.172392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:04.172392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:04.172392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:04.172392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:04.172392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:04.172392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:04.172392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:04.172392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:04.172392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:04.172392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:04.172392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:04.172392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:04.172392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:04.172392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:04.173436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:04.173436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:04.173436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:04.173436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:04.173436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:04.173436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:04.173436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:04.173436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:04.173436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:04.173436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:04.173436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:04.173436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:04.173436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:04.173436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:04.173436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:04.173436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:04.173436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:04.174355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:04.174355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:04.174355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:04.174355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:04.174355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:04.174355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:04.174355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:04.174355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:04.183463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 306/606] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:04.190284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2da3d7-0798-4c68-845b-e0bfd4a70184 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:04.298529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 301/607] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:53:04.299888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9e95a-df70-46a6-a73f-d3c7f234c069 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:04.300182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9e95a-df70-46a6-a73f-d3c7f234c069 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:04.300463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9e95a-df70-46a6-a73f-d3c7f234c069 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:04.306450 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 302/608] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:04.308068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606836d-b00c-4b3d-8a41-74441bfc1745 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:04.308360 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606836d-b00c-4b3d-8a41-74441bfc1745 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:04.308645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606836d-b00c-4b3d-8a41-74441bfc1745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:04.323639 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2da3d7-0798-4c68-845b-e0bfd4a70184 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:04.326932 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-1606836d-b00c-4b3d-8a41-74441bfc1745 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:04.328137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1606836d-b00c-4b3d-8a41-74441bfc1745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:04.359689 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a1431-0fff-430d-864a-dfe2c418ae39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:04.360350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a1431-0fff-430d-864a-dfe2c418ae39 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:04.360636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a1431-0fff-430d-864a-dfe2c418ae39 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:04.360940 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a1431-0fff-430d-864a-dfe2c418ae39 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:04.361728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a1431-0fff-430d-864a-dfe2c418ae39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:04.362037 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a1431-0fff-430d-864a-dfe2c418ae39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:04.363460 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a1431-0fff-430d-864a-dfe2c418ae39 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:04.377161 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 307/609] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:04.378009 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a1431-0fff-430d-864a-dfe2c418ae39 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:04.398036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:04.408912 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 303/610] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:53:04.409803 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606836d-b00c-4b3d-8a41-74441bfc1745 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:04.487380 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a1431-0fff-430d-864a-dfe2c418ae39 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:04.505066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:04.524842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b277c-b043-4d6a-8192-881f41f7208b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:04.525644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b277c-b043-4d6a-8192-881f41f7208b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:04.525963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b277c-b043-4d6a-8192-881f41f7208b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:04.526200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b277c-b043-4d6a-8192-881f41f7208b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:04.536952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b277c-b043-4d6a-8192-881f41f7208b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:04.542338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606836d-b00c-4b3d-8a41-74441bfc1745 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:04.545868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-794b277c-b043-4d6a-8192-881f41f7208b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:04.545868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:04.545868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:04.545868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:04.545868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:04.545868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:04.545868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:04.545868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:04.545868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:04.545868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:04.545868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:04.545868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:04.545868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:04.545868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:04.545868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:04.545868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:04.546813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:04.546813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:04.546813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:04.546813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:04.546813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:04.546813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:04.546813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:04.546813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:04.546813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:04.546813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:04.546813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:04.546813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:04.546813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:04.546813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:04.546813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:04.546813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:04.547640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:04.547640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:04.547640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:04.547640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:04.547640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:04.547640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:04.547640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:04.547640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:04.547640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:04.547640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:04.547640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:04.547640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:04.547640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 308/611] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:53:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:04.548323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b277c-b043-4d6a-8192-881f41f7208b None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:04.580702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654669f-f2a5-4204-96f3-63f7528c6bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:04.582193 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654669f-f2a5-4204-96f3-63f7528c6bb8 None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:04.582607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654669f-f2a5-4204-96f3-63f7528c6bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:04.583021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654669f-f2a5-4204-96f3-63f7528c6bb8 None admin] PATH_INFO: `/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/53b891a734a64b5983564e207de0359d/roles/f1e42ea5a1c440bba915b459d2744858` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:04.603378 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654669f-f2a5-4204-96f3-63f7528c6bb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=459e4743ff3043f69408ca2095be3bd0, user_id=53b891a734a64b5983564e207de0359d, role_id=f1e42ea5a1c440bba915b459d2744858)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:04.605184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654669f-f2a5-4204-96f3-63f7528c6bb8 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:04.626975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 304/612] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:53:04 2022] PUT /identity/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/53b891a734a64b5983564e207de0359d/roles/f1e42ea5a1c440bba915b459d2744858 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:53:04.627904 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654669f-f2a5-4204-96f3-63f7528c6bb8 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:04.647948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:04.677331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b277c-b043-4d6a-8192-881f41f7208b None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:04.736037 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6d4d0-5e8f-42a2-b222-f7cfdff32f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:04.736764 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6d4d0-5e8f-42a2-b222-f7cfdff32f67 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:04.737057 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6d4d0-5e8f-42a2-b222-f7cfdff32f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:04.737337 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6d4d0-5e8f-42a2-b222-f7cfdff32f67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:04.757367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6d4d0-5e8f-42a2-b222-f7cfdff32f67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:04.767635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-fde6d4d0-5e8f-42a2-b222-f7cfdff32f67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:04.767635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:04.767635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:04.767635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:04.767635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:04.767635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:04.767635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:04.767635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:04.767635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:04.767635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:04.767635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:04.767635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:04.767635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:04.767635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:04.767635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:04.767635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:04.768884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:04.768884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:04.768884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:04.768884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:04.768884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:04.768884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:04.768884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:04.768884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:04.768884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:04.768884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:04.768884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:04.768884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:04.768884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:04.768884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:04.768884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:04.768884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:04.769895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:04.769895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:04.769895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:04.769895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:04.769895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:04.769895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:04.769895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:04.769895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:04.769895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:04.769895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:04.769895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:04.769895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:04.769895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 309/613] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:53:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:04.776855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6d4d0-5e8f-42a2-b222-f7cfdff32f67 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:04.784142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654669f-f2a5-4204-96f3-63f7528c6bb8 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:04.786884 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:04.843166 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de2c16-fa3e-46a0-9837-b7638d801043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:04.843782 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de2c16-fa3e-46a0-9837-b7638d801043 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:04.844067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de2c16-fa3e-46a0-9837-b7638d801043 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:04.844346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de2c16-fa3e-46a0-9837-b7638d801043 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:04.845139 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de2c16-fa3e-46a0-9837-b7638d801043 None admin] RBAC: Adding query filter params (name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:04.845452 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de2c16-fa3e-46a0-9837-b7638d801043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:04.849904 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de2c16-fa3e-46a0-9837-b7638d801043 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:04.869602 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 305/614] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:53:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:04.876155 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de2c16-fa3e-46a0-9837-b7638d801043 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:04.944208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6d4d0-5e8f-42a2-b222-f7cfdff32f67 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:04.991962 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6179e-89d0-4e5c-96cd-c6a294c2fb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:04.993095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6179e-89d0-4e5c-96cd-c6a294c2fb5e None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:04.994453 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6179e-89d0-4e5c-96cd-c6a294c2fb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:04.994848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6179e-89d0-4e5c-96cd-c6a294c2fb5e None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:04.995796 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6179e-89d0-4e5c-96cd-c6a294c2fb5e None admin] RBAC: Adding query filter params (name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:04.996239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6179e-89d0-4e5c-96cd-c6a294c2fb5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:04.997924 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6179e-89d0-4e5c-96cd-c6a294c2fb5e None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:05.004188 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:05.010894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 310/615] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:53:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:05.019194 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6179e-89d0-4e5c-96cd-c6a294c2fb5e None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:05.059854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de2c16-fa3e-46a0-9837-b7638d801043 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:05.132302 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e08e3-d0a2-42d1-9af6-df7b3982f1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:05.134289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e08e3-d0a2-42d1-9af6-df7b3982f1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:05.134580 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e08e3-d0a2-42d1-9af6-df7b3982f1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:05.134860 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e08e3-d0a2-42d1-9af6-df7b3982f1b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:05.158197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e08e3-d0a2-42d1-9af6-df7b3982f1b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:05.159692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e08e3-d0a2-42d1-9af6-df7b3982f1b0 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:05.192622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-a18e08e3-d0a2-42d1-9af6-df7b3982f1b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:05.192622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:05.192622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:05.192622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:05.192622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:05.192622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:05.192622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:05.192622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:05.192622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:05.192622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:05.192622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:05.192622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:05.192622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:05.192622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:05.192622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:05.193779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:05.193779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:05.193779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:05.193779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:05.193779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:05.193779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:05.193779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:05.193779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:05.193779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:05.193779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:05.193779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:05.193779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:05.193779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:05.193779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:05.193779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:05.193779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:05.193779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:05.194856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:05.194856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:05.194856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:05.194856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:05.194856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 306/616] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:05.202339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e08e3-d0a2-42d1-9af6-df7b3982f1b0 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:05.219194 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6179e-89d0-4e5c-96cd-c6a294c2fb5e None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:05.236261 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:05.296672 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54caaa-55e9-497f-ba18-e3b4f29ebfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:05.297815 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54caaa-55e9-497f-ba18-e3b4f29ebfd8 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:05.298383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54caaa-55e9-497f-ba18-e3b4f29ebfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:05.299584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54caaa-55e9-497f-ba18-e3b4f29ebfd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:05.321043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54caaa-55e9-497f-ba18-e3b4f29ebfd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:05.322738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54caaa-55e9-497f-ba18-e3b4f29ebfd8 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:05.342576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-9d54caaa-55e9-497f-ba18-e3b4f29ebfd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:05.342576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:05.342576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:05.342576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:05.342576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:05.342576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:05.342576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:05.342576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:05.342576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:05.342576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:05.342576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:05.342576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:05.342576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:05.342576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:05.342576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:05.343952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:05.343952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:05.343952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:05.343952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:05.343952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:05.343952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:05.343952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:05.343952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:05.343952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:05.343952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:05.343952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:05.343952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:05.343952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:05.343952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:05.343952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:05.343952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:05.343952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:05.344860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:05.344860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:05.344860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:05.344860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:05.347986 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 311/617] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:05.353897 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54caaa-55e9-497f-ba18-e3b4f29ebfd8 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:05.378969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e08e3-d0a2-42d1-9af6-df7b3982f1b0 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:05.413156 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:05.453457 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d77be-a4df-4b80-81bf-11ff06bfe836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:05.454419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d77be-a4df-4b80-81bf-11ff06bfe836 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:05.454419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d77be-a4df-4b80-81bf-11ff06bfe836 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:05.454716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d77be-a4df-4b80-81bf-11ff06bfe836 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:05.455435 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d77be-a4df-4b80-81bf-11ff06bfe836 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:05.455798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d77be-a4df-4b80-81bf-11ff06bfe836 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:05.457176 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d77be-a4df-4b80-81bf-11ff06bfe836 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:05.469600 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54caaa-55e9-497f-ba18-e3b4f29ebfd8 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:05.486934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 307/618] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:05.497834 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d77be-a4df-4b80-81bf-11ff06bfe836 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:05.524136 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886858b-be5b-4d56-8b1c-16102bace338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:05.525308 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886858b-be5b-4d56-8b1c-16102bace338 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:05.526236 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886858b-be5b-4d56-8b1c-16102bace338 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:05.526921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886858b-be5b-4d56-8b1c-16102bace338 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:05.528565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6886858b-be5b-4d56-8b1c-16102bace338 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:05.529119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6886858b-be5b-4d56-8b1c-16102bace338 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:05.530973 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6886858b-be5b-4d56-8b1c-16102bace338 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:05.544192 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 312/619] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:05.550021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886858b-be5b-4d56-8b1c-16102bace338 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:05.699096 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:05.710097 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886858b-be5b-4d56-8b1c-16102bace338 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:05.737063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:05.741167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d77be-a4df-4b80-81bf-11ff06bfe836 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:05.777611 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e11b6f-1b05-4fef-a419-ef32439be4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:05.778445 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e11b6f-1b05-4fef-a419-ef32439be4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:05.778734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e11b6f-1b05-4fef-a419-ef32439be4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:05.779012 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e11b6f-1b05-4fef-a419-ef32439be4c6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:05.791511 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e11b6f-1b05-4fef-a419-ef32439be4c6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:05.793249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e11b6f-1b05-4fef-a419-ef32439be4c6 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:05.819332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-17e11b6f-1b05-4fef-a419-ef32439be4c6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 21:53:05.819332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:05.819332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:05.819332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:05.819332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:05.819332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:05.819332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:05.819332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:05.819332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:05.819332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:05.819332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:05.819332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:05.819332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:05.819332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:05.819332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:05.819332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:05.821055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:05.822166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:05.822166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:05.822166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:05.822166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:05.822166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:05.822166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:05.822166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:05.822166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:05.822166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 21:53:05.822166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:05.824733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c0e24-c7a4-4011-98b0-996e2d8df356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:05.825727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c0e24-c7a4-4011-98b0-996e2d8df356 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:05.826159 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c0e24-c7a4-4011-98b0-996e2d8df356 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:05.826590 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c0e24-c7a4-4011-98b0-996e2d8df356 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:05.834690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c0e24-c7a4-4011-98b0-996e2d8df356 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:05.837777 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c0e24-c7a4-4011-98b0-996e2d8df356 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:05.838218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 313/620] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:53:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:05.845636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e11b6f-1b05-4fef-a419-ef32439be4c6 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:05.858476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-387c0e24-c7a4-4011-98b0-996e2d8df356 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 21:53:05.858476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:05.858476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:05.858476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:05.858476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:05.858476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:05.858476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:05.858476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:05.858476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:05.858476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:05.858476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:05.858476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:05.858476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:05.858476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:05.858476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:05.858476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:05.859588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:05.860765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:05.860765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:05.860765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:05.860765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:05.860765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:05.860765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:05.860765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:05.860765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:05.860765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 21:53:05.860765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:05.862727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 308/621] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:53:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:05.873689 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c0e24-c7a4-4011-98b0-996e2d8df356 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:05.968138 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e11b6f-1b05-4fef-a419-ef32439be4c6 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:05.994107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c0e24-c7a4-4011-98b0-996e2d8df356 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:06.018866 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258777bd-3068-4f28-9358-f93eabe2bb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:06.019975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258777bd-3068-4f28-9358-f93eabe2bb86 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:06.020468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258777bd-3068-4f28-9358-f93eabe2bb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:06.020971 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258777bd-3068-4f28-9358-f93eabe2bb86 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:06.022044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258777bd-3068-4f28-9358-f93eabe2bb86 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:06.022565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258777bd-3068-4f28-9358-f93eabe2bb86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:06.024751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258777bd-3068-4f28-9358-f93eabe2bb86 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:06.038321 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a154a23-d2a3-4287-af54-e875ba1fa75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:06.039173 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a154a23-d2a3-4287-af54-e875ba1fa75a None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:06.039457 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a154a23-d2a3-4287-af54-e875ba1fa75a None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:06.039919 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a154a23-d2a3-4287-af54-e875ba1fa75a None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:06.040815 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a154a23-d2a3-4287-af54-e875ba1fa75a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:06.041133 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a154a23-d2a3-4287-af54-e875ba1fa75a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:06.042582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a154a23-d2a3-4287-af54-e875ba1fa75a None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:06.052312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:06.065515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 309/622] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Jul 12 21:53:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:06.069414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 314/623] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Jul 12 21:53:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:06.071705 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a154a23-d2a3-4287-af54-e875ba1fa75a None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:06.076675 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258777bd-3068-4f28-9358-f93eabe2bb86 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:06.142907 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:06.245091 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258777bd-3068-4f28-9358-f93eabe2bb86 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:06.246657 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a154a23-d2a3-4287-af54-e875ba1fa75a None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:06.282882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef5001-bba6-4100-b5c6-921da5ad158b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:06.283555 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef5001-bba6-4100-b5c6-921da5ad158b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:06.283842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef5001-bba6-4100-b5c6-921da5ad158b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:06.284120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef5001-bba6-4100-b5c6-921da5ad158b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:06.290922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4be98-033d-4683-af1f-ef5ef6b26052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:06.291722 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4be98-033d-4683-af1f-ef5ef6b26052 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:06.292002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4be98-033d-4683-af1f-ef5ef6b26052 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:06.292274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4be98-033d-4683-af1f-ef5ef6b26052 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:06.307130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4be98-033d-4683-af1f-ef5ef6b26052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:06.308596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4be98-033d-4683-af1f-ef5ef6b26052 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:06.331690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-d4e4be98-033d-4683-af1f-ef5ef6b26052 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:06.331690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:06.331690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:06.331690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:06.331690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:06.331690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:06.331690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:06.331690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:06.331690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:06.331690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:06.331690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:06.331690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:06.331690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:06.331690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:06.331690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:06.332621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:06.332621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:06.332621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:06.332621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:06.332621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:06.332621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:06.332621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:06.332621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:06.332621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:06.332621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:06.332621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:06.332621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:06.332621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:06.332621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:06.332621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:06.332621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:06.332621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:06.333581 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:06.333581 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:06.333581 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:06.333581 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:06.339284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 310/624] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:06.340653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5770d-e0dc-466e-b23a-a6992b249ba8 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:06.340971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5770d-e0dc-466e-b23a-a6992b249ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:06.341254 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5770d-e0dc-466e-b23a-a6992b249ba8 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:06.345062 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 311/625] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:06.345841 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5770d-e0dc-466e-b23a-a6992b249ba8 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:06.367564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef5001-bba6-4100-b5c6-921da5ad158b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:06.369599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fef5001-bba6-4100-b5c6-921da5ad158b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:06.403539 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:06.410120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:06.463967 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5770d-e0dc-466e-b23a-a6992b249ba8 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:06.504521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae7e19-b621-4e07-80ed-e7e5ebcae59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:06.505299 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae7e19-b621-4e07-80ed-e7e5ebcae59f None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:06.505657 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae7e19-b621-4e07-80ed-e7e5ebcae59f None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:06.506118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae7e19-b621-4e07-80ed-e7e5ebcae59f None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:06.506951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae7e19-b621-4e07-80ed-e7e5ebcae59f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:06.507370 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae7e19-b621-4e07-80ed-e7e5ebcae59f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:06.509579 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae7e19-b621-4e07-80ed-e7e5ebcae59f None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:06.515054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 315/626] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:53:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:53:06.515054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d877d-6d81-4cf0-904b-11b0e691d285 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:06.515054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d877d-6d81-4cf0-904b-11b0e691d285 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:06.515054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d877d-6d81-4cf0-904b-11b0e691d285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:06.528894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 312/627] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:06.530155 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae7e19-b621-4e07-80ed-e7e5ebcae59f None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:06.537592 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-1f0d877d-6d81-4cf0-904b-11b0e691d285 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:06.538817 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-1f0d877d-6d81-4cf0-904b-11b0e691d285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:06.621767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 316/628] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:53:06.622961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d877d-6d81-4cf0-904b-11b0e691d285 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:06.653480 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae7e19-b621-4e07-80ed-e7e5ebcae59f None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:06.663214 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:06.692430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:06.693915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46941258-6be7-4a8d-8d39-053d20c47d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:06.694572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46941258-6be7-4a8d-8d39-053d20c47d74 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:06.694801 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46941258-6be7-4a8d-8d39-053d20c47d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:06.695721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46941258-6be7-4a8d-8d39-053d20c47d74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:06.706695 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46941258-6be7-4a8d-8d39-053d20c47d74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:06.708154 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46941258-6be7-4a8d-8d39-053d20c47d74 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:06.726965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-46941258-6be7-4a8d-8d39-053d20c47d74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:06.726965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:06.726965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:06.726965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:06.726965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:06.726965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:06.726965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:06.726965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:06.726965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:06.726965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:06.726965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:06.726965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:06.726965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:06.726965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:06.726965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:06.728190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:06.728190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:06.728190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:06.728190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:06.728190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:06.728190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:06.728190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:06.728190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:06.728190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:06.728190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:06.728190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:06.728190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:06.728190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:06.728190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:06.728190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:06.728190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:06.728190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:06.729799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:06.729799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:06.729799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:06.729799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:06.729799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 313/629] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:06.729799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdd8a1-053e-496e-ae10-e7c134f5ae1d None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:06.730154 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdd8a1-053e-496e-ae10-e7c134f5ae1d None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:06.730656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdd8a1-053e-496e-ae10-e7c134f5ae1d None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:06.731717 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 314/630] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:06.733528 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdd8a1-053e-496e-ae10-e7c134f5ae1d None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:06.746414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d877d-6d81-4cf0-904b-11b0e691d285 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:06.783600 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08d1e7-2a96-4706-b547-f6a9f2e607dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:06.784687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08d1e7-2a96-4706-b547-f6a9f2e607dc None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:06.785005 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08d1e7-2a96-4706-b547-f6a9f2e607dc None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:06.785284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08d1e7-2a96-4706-b547-f6a9f2e607dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:06.793140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08d1e7-2a96-4706-b547-f6a9f2e607dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:06.794797 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08d1e7-2a96-4706-b547-f6a9f2e607dc None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:06.809855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-bf08d1e7-2a96-4706-b547-f6a9f2e607dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:06.809855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:06.809855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:06.809855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:06.809855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:06.809855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:06.809855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:06.809855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:06.809855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:06.809855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:06.809855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:06.809855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:06.809855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:06.809855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:06.809855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:06.810799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:06.810799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:06.810799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:06.810799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:06.810799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:06.810799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:06.810799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:06.810799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:06.810799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:06.810799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:06.810799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:06.810799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:06.810799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:06.810799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:06.810799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:06.810799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:06.810799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:06.811669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:06.811669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:06.811669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:06.811669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:06.811669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:06.811669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:06.811669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:06.811669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:06.811669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 317/631] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:06.812156 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08d1e7-2a96-4706-b547-f6a9f2e607dc None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:06.868116 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdd8a1-053e-496e-ae10-e7c134f5ae1d None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:06.906696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad75075-39d5-4da7-ae07-84a12eef7514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:06.907803 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad75075-39d5-4da7-ae07-84a12eef7514 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:06.908271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad75075-39d5-4da7-ae07-84a12eef7514 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:06.908867 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad75075-39d5-4da7-ae07-84a12eef7514 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:06.923749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad75075-39d5-4da7-ae07-84a12eef7514 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:06.929245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08d1e7-2a96-4706-b547-f6a9f2e607dc None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:06.932563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-2ad75075-39d5-4da7-ae07-84a12eef7514 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:06.932563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:06.932563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:06.932563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:06.932563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:06.932563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:06.932563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:06.932563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:06.932563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:06.932563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:06.932563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:06.932563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:06.932563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:06.932563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:06.932563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:06.932563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:06.933656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:06.933656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:06.933656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:06.933656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:06.933656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:06.933656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:06.933656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:06.933656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:06.933656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:06.933656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:06.933656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:06.933656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:06.933656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:06.933656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:06.933656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:06.933656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:06.934673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:06.934673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:06.934673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:06.934673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:06.934673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:06.934673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:06.934673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:06.934673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:06.934673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:06.934673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:06.934673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:06.934673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:06.937180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 315/632] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:53:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:06.938697 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69acb6a-8479-4e51-b6a9-6d565ddc8026 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:06.938982 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69acb6a-8479-4e51-b6a9-6d565ddc8026 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:06.939258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69acb6a-8479-4e51-b6a9-6d565ddc8026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:06.954260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:06.959029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-a69acb6a-8479-4e51-b6a9-6d565ddc8026 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:06.960243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a69acb6a-8479-4e51-b6a9-6d565ddc8026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:06.975189 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c85d0-d052-493d-bbea-dbab923a13d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:06.975857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c85d0-d052-493d-bbea-dbab923a13d0 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:06.976141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c85d0-d052-493d-bbea-dbab923a13d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:06.977164 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c85d0-d052-493d-bbea-dbab923a13d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:06.978033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c85d0-d052-493d-bbea-dbab923a13d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:06.978350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c85d0-d052-493d-bbea-dbab923a13d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:06.979212 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:06.979728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c85d0-d052-493d-bbea-dbab923a13d0 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:06.996768 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 318/633] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:06.996768 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c85d0-d052-493d-bbea-dbab923a13d0 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:07.055407 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 316/634] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:53:07.057117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69acb6a-8479-4e51-b6a9-6d565ddc8026 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:07.108822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c85d0-d052-493d-bbea-dbab923a13d0 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:07.149295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f089a7d-0e51-4231-b548-4bd8a2d3ee62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:07.149957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f089a7d-0e51-4231-b548-4bd8a2d3ee62 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:07.150234 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f089a7d-0e51-4231-b548-4bd8a2d3ee62 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:07.150503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f089a7d-0e51-4231-b548-4bd8a2d3ee62 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:07.151287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f089a7d-0e51-4231-b548-4bd8a2d3ee62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:07.151590 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f089a7d-0e51-4231-b548-4bd8a2d3ee62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:07.153056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f089a7d-0e51-4231-b548-4bd8a2d3ee62 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:07.164810 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 319/635] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:07.165646 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f089a7d-0e51-4231-b548-4bd8a2d3ee62 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:07.183732 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69acb6a-8479-4e51-b6a9-6d565ddc8026 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:07.208059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:07.219659 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f49138-acac-468c-a6f0-cad3100ff29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:07.220332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f49138-acac-468c-a6f0-cad3100ff29f None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:07.220612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f49138-acac-468c-a6f0-cad3100ff29f None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:07.220915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f49138-acac-468c-a6f0-cad3100ff29f None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:07.221662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f49138-acac-468c-a6f0-cad3100ff29f None admin] RBAC: Adding query filter params (name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:07.221969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f49138-acac-468c-a6f0-cad3100ff29f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:07.223347 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f49138-acac-468c-a6f0-cad3100ff29f None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:07.231960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 317/636] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:53:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:07.232799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f49138-acac-468c-a6f0-cad3100ff29f None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:07.250200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:07.274934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f089a7d-0e51-4231-b548-4bd8a2d3ee62 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:07.311063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be54144-db9e-450f-bb9a-cbd384a86baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:07.311775 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be54144-db9e-450f-bb9a-cbd384a86baa None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:07.312062 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be54144-db9e-450f-bb9a-cbd384a86baa None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:07.312343 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be54144-db9e-450f-bb9a-cbd384a86baa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:07.319882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be54144-db9e-450f-bb9a-cbd384a86baa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:07.321760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be54144-db9e-450f-bb9a-cbd384a86baa None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:07.337210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-7be54144-db9e-450f-bb9a-cbd384a86baa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:07.337210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:07.337210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:07.337210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:07.337210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:07.337210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:07.337210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:07.337210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:07.337210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:07.337210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:07.337210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:07.337210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:07.337210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:07.337210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:07.337210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:07.338270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:07.338270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:07.338270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:07.338270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:07.338270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:07.338270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:07.338270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:07.338270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:07.338270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:07.338270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:07.338270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:07.338270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:07.338270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:07.338270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:07.338270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:07.338270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:07.338270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:07.339280 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:07.339280 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:07.339280 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:07.339280 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:07.339280 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:07.339280 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:07.339280 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:07.339280 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:07.339280 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 320/637] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:07.339791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4c758-f278-4c74-9602-41a87e013fa9 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:07.340637 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4c758-f278-4c74-9602-41a87e013fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:07.340637 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4c758-f278-4c74-9602-41a87e013fa9 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:07.342186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 321/638] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:07.343398 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f49138-acac-468c-a6f0-cad3100ff29f None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:07.344938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4c758-f278-4c74-9602-41a87e013fa9 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:07.382381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101df2a-f451-43b5-9997-bc335cf1fb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:07.383187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101df2a-f451-43b5-9997-bc335cf1fb40 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:07.383472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101df2a-f451-43b5-9997-bc335cf1fb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:07.383753 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101df2a-f451-43b5-9997-bc335cf1fb40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:07.394447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101df2a-f451-43b5-9997-bc335cf1fb40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:07.402548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-4101df2a-f451-43b5-9997-bc335cf1fb40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:07.402548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:07.402548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:07.402548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:07.402548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:07.402548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:07.402548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:07.402548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:07.402548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:07.402548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:07.402548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:07.402548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:07.402548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:07.402548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:07.402548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:07.402548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:07.403523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:07.403523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:07.403523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:07.403523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:07.403523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:07.403523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:07.403523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:07.403523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:07.403523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:07.403523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:07.403523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:07.403523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:07.403523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:07.403523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:07.403523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:07.403523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:07.404830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:07.404830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:07.404830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:07.404830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:07.404830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:07.404830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:07.404830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:07.404830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:07.404830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:07.404830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:07.404830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:07.404830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:07.404830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 318/639] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:53:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:07.404830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101df2a-f451-43b5-9997-bc335cf1fb40 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:07.455310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:07.460280 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4c758-f278-4c74-9602-41a87e013fa9 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:07.479308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:07.507432 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e0d61-f88f-45a4-b98e-8406b1be0846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:07.508999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e0d61-f88f-45a4-b98e-8406b1be0846 None admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:07.509294 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e0d61-f88f-45a4-b98e-8406b1be0846 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:07.509571 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e0d61-f88f-45a4-b98e-8406b1be0846 None admin] PATH_INFO: `/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/837bb5ac18554dfdbbcc776689639494/roles/f1e42ea5a1c440bba915b459d2744858` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:07.516430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101df2a-f451-43b5-9997-bc335cf1fb40 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:07.527409 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e0d61-f88f-45a4-b98e-8406b1be0846 None admin] RBAC: Authorizing `identity:create_grant(project_id=459e4743ff3043f69408ca2095be3bd0, user_id=837bb5ac18554dfdbbcc776689639494, role_id=f1e42ea5a1c440bba915b459d2744858)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:07.528981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e0d61-f88f-45a4-b98e-8406b1be0846 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:07.541843 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 322/640] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:53:07 2022] PUT /identity/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/837bb5ac18554dfdbbcc776689639494/roles/f1e42ea5a1c440bba915b459d2744858 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:53:07.542973 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e0d61-f88f-45a4-b98e-8406b1be0846 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:07.566954 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452d000-9b6a-4f7d-a387-f868715e1cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:07.567709 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452d000-9b6a-4f7d-a387-f868715e1cab None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:07.567995 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452d000-9b6a-4f7d-a387-f868715e1cab None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:07.568330 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452d000-9b6a-4f7d-a387-f868715e1cab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:07.581560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452d000-9b6a-4f7d-a387-f868715e1cab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:07.583609 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452d000-9b6a-4f7d-a387-f868715e1cab None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:07.604042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-9452d000-9b6a-4f7d-a387-f868715e1cab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:07.604042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:07.604042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:07.604042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:07.604042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:07.604042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:07.604042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:07.604042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:07.604042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:07.604042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:07.604042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:07.604042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:07.604042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:07.604042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:07.604042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:07.605427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:07.605427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:07.605427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:07.605427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:07.605427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:07.605427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:07.605427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:07.605427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:07.605427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:07.605427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:07.605427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:07.605427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:07.605427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:07.605427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:07.605427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:07.605427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:07.605427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:07.606834 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:07.606834 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:07.606834 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:07.606834 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:07.606834 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 319/641] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:07.607168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123eeb63-55ed-45ee-8b78-52196b09a975 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:07.607168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123eeb63-55ed-45ee-8b78-52196b09a975 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:07.607540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123eeb63-55ed-45ee-8b78-52196b09a975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:07.628139 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-123eeb63-55ed-45ee-8b78-52196b09a975 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:07.629799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-123eeb63-55ed-45ee-8b78-52196b09a975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:07.681917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e0d61-f88f-45a4-b98e-8406b1be0846 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:07.710286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 320/642] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:53:07.711574 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123eeb63-55ed-45ee-8b78-52196b09a975 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:07.728982 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b357ecc-6daf-45e0-9338-3888d7da4a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:07.729664 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b357ecc-6daf-45e0-9338-3888d7da4a4d None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:07.729949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b357ecc-6daf-45e0-9338-3888d7da4a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:07.730229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b357ecc-6daf-45e0-9338-3888d7da4a4d None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:07.731029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b357ecc-6daf-45e0-9338-3888d7da4a4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:07.731343 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b357ecc-6daf-45e0-9338-3888d7da4a4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:07.732850 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b357ecc-6daf-45e0-9338-3888d7da4a4d None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:07.745734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 323/643] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:07.746570 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b357ecc-6daf-45e0-9338-3888d7da4a4d None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:07.759275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:07.778253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:07.825776 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123eeb63-55ed-45ee-8b78-52196b09a975 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:07.862258 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b357ecc-6daf-45e0-9338-3888d7da4a4d None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:07.863247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767af3a-c8d7-483f-9611-90a1785c43a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:07.863920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767af3a-c8d7-483f-9611-90a1785c43a3 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:07.864200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767af3a-c8d7-483f-9611-90a1785c43a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:07.864472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767af3a-c8d7-483f-9611-90a1785c43a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:07.865274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767af3a-c8d7-483f-9611-90a1785c43a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:07.865579 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767af3a-c8d7-483f-9611-90a1785c43a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:07.866992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767af3a-c8d7-483f-9611-90a1785c43a3 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:07.875533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 321/644] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:53:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:07.876357 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767af3a-c8d7-483f-9611-90a1785c43a3 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:07.905288 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51895ade-059c-473f-a07e-bbef778a3c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:07.905944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51895ade-059c-473f-a07e-bbef778a3c7c None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:07.906226 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51895ade-059c-473f-a07e-bbef778a3c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:07.906502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51895ade-059c-473f-a07e-bbef778a3c7c None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:07.907220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51895ade-059c-473f-a07e-bbef778a3c7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:07.907528 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51895ade-059c-473f-a07e-bbef778a3c7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:07.909330 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51895ade-059c-473f-a07e-bbef778a3c7c None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:07.926615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 324/645] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:07.926615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51895ade-059c-473f-a07e-bbef778a3c7c None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:07.998354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767af3a-c8d7-483f-9611-90a1785c43a3 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:08.009249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:08.020631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:08.040867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51895ade-059c-473f-a07e-bbef778a3c7c None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:08.042559 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50785c3e-7531-4f4e-ba6b-00a8399209c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:08.043369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50785c3e-7531-4f4e-ba6b-00a8399209c4 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:08.043652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50785c3e-7531-4f4e-ba6b-00a8399209c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:08.044001 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50785c3e-7531-4f4e-ba6b-00a8399209c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:08.056702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50785c3e-7531-4f4e-ba6b-00a8399209c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:08.064410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-50785c3e-7531-4f4e-ba6b-00a8399209c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:08.064410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:08.064410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:08.064410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:08.064410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:08.064410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:08.064410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:08.064410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:08.064410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:08.064410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:08.064410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:08.064410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:08.064410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:08.064410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:08.064410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:08.064410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:08.065521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:08.065521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:08.065521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:08.065521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:08.065521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:08.065521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:08.065521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:08.065521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:08.065521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:08.065521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:08.065521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:08.065521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:08.065521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:08.065521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:08.065521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:08.065521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:08.066789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:08.066789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:08.066789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:08.066789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:08.066789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:08.066789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:08.066789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:08.066789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:08.066789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:08.066789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:08.066789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:08.066789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:08.066789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 322/646] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:53:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:08.067527 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50785c3e-7531-4f4e-ba6b-00a8399209c4 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:08.083004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ef938-4196-4f96-bf96-dc7d0a45f74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:08.083890 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ef938-4196-4f96-bf96-dc7d0a45f74c None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:08.084331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ef938-4196-4f96-bf96-dc7d0a45f74c None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:08.084815 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ef938-4196-4f96-bf96-dc7d0a45f74c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:08.085671 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ef938-4196-4f96-bf96-dc7d0a45f74c None admin] RBAC: Adding query filter params (role.id=f1e42ea5a1c440bba915b459d2744858, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=c53e1e801e5e40e794e4ccfff34ec988) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:08.086574 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ef938-4196-4f96-bf96-dc7d0a45f74c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:08.088009 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ef938-4196-4f96-bf96-dc7d0a45f74c None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:08.104743 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 325/647] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:53:07 2022] GET /identity/v3/role_assignments?user.id=c53e1e801e5e40e794e4ccfff34ec988&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=f1e42ea5a1c440bba915b459d2744858 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:08.104743 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ef938-4196-4f96-bf96-dc7d0a45f74c None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:08.190066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50785c3e-7531-4f4e-ba6b-00a8399209c4 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:08.241557 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34276838-609e-444b-93d1-14667c281eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:08.243024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34276838-609e-444b-93d1-14667c281eca None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:08.243435 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34276838-609e-444b-93d1-14667c281eca None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:08.244114 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34276838-609e-444b-93d1-14667c281eca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:08.251276 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ef938-4196-4f96-bf96-dc7d0a45f74c None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:08.260752 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34276838-609e-444b-93d1-14667c281eca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:08.263903 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34276838-609e-444b-93d1-14667c281eca None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:08.285713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-34276838-609e-444b-93d1-14667c281eca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:08.285713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:08.285713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:08.285713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:08.285713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:08.285713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:08.285713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:08.285713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:08.285713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:08.285713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:08.285713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:08.285713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:08.285713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:08.285713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:08.285713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:08.286685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:08.286685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:08.286685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:08.286685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:08.286685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:08.286685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:08.286685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:08.286685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:08.286685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:08.286685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:08.286685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:08.286685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:08.286685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:08.286685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:08.286685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:08.286685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:08.286685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:08.287567 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:08.287567 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:08.287567 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:08.287567 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:08.292220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e0e19-e728-4df7-84f8-f14b61910475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:08.292932 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 323/648] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:08.293218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e0e19-e728-4df7-84f8-f14b61910475 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:08.293515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e0e19-e728-4df7-84f8-f14b61910475 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:08.293798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e0e19-e728-4df7-84f8-f14b61910475 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:08.294319 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34276838-609e-444b-93d1-14667c281eca None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:08.296261 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:08.303668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e0e19-e728-4df7-84f8-f14b61910475 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:08.305294 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e0e19-e728-4df7-84f8-f14b61910475 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:08.323582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-558e0e19-e728-4df7-84f8-f14b61910475 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 21:53:08.323582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:08.323582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:08.323582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:08.323582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:08.323582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:08.323582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:08.323582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:08.323582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:08.323582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:08.323582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:08.323582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:08.323582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:08.323582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:08.323582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:08.323582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:08.324879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:08.326059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:08.326059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:08.326059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:08.326059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:08.326059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:08.326059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:08.326059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:08.326059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:08.326059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 21:53:08.326059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:08.326897 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 326/649] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:53:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:08.331317 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:08.332803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e0e19-e728-4df7-84f8-f14b61910475 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:08.427045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34276838-609e-444b-93d1-14667c281eca None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:08.452615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e0e19-e728-4df7-84f8-f14b61910475 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:08.466211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09dbbfe-346e-4bde-b11a-de9cd8cd6d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:08.466955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dbbfe-346e-4bde-b11a-de9cd8cd6d11 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:08.467341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dbbfe-346e-4bde-b11a-de9cd8cd6d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:08.467625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dbbfe-346e-4bde-b11a-de9cd8cd6d11 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:08.468651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09dbbfe-346e-4bde-b11a-de9cd8cd6d11 None admin] RBAC: Adding query filter params (name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:08.468996 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09dbbfe-346e-4bde-b11a-de9cd8cd6d11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:08.470418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09dbbfe-346e-4bde-b11a-de9cd8cd6d11 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:08.480521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 324/650] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:53:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:08.481486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09dbbfe-346e-4bde-b11a-de9cd8cd6d11 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:08.493419 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd066a8-e5a1-4904-8d50-a84c4fa09e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:08.494089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd066a8-e5a1-4904-8d50-a84c4fa09e88 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:08.494372 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd066a8-e5a1-4904-8d50-a84c4fa09e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:08.494647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd066a8-e5a1-4904-8d50-a84c4fa09e88 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:08.495379 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd066a8-e5a1-4904-8d50-a84c4fa09e88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:08.495686 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd066a8-e5a1-4904-8d50-a84c4fa09e88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:08.497105 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd066a8-e5a1-4904-8d50-a84c4fa09e88 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:08.509527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 327/651] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:08.510384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd066a8-e5a1-4904-8d50-a84c4fa09e88 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:08.572125 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:08.591310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:08.602973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09dbbfe-346e-4bde-b11a-de9cd8cd6d11 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:08.619793 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd066a8-e5a1-4904-8d50-a84c4fa09e88 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:08.642455 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ee0a8-f605-409f-b389-a45a4e4a3fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:08.643500 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ee0a8-f605-409f-b389-a45a4e4a3fbd None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:08.644053 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ee0a8-f605-409f-b389-a45a4e4a3fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:08.644491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ee0a8-f605-409f-b389-a45a4e4a3fbd None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:08.645576 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ee0a8-f605-409f-b389-a45a4e4a3fbd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:08.646180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ee0a8-f605-409f-b389-a45a4e4a3fbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:08.647940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ee0a8-f605-409f-b389-a45a4e4a3fbd None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:08.654948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4c537-9e24-43af-acec-9a3bac20468b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:08.655743 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4c537-9e24-43af-acec-9a3bac20468b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:08.656021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4c537-9e24-43af-acec-9a3bac20468b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:08.656292 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4c537-9e24-43af-acec-9a3bac20468b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:08.673882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4c537-9e24-43af-acec-9a3bac20468b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:08.673882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4c537-9e24-43af-acec-9a3bac20468b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:08.674353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 325/652] 10.222.0.44 () {56 vars in 1137 bytes} [Tue Jul 12 21:53:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:08.675635 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ee0a8-f605-409f-b389-a45a4e4a3fbd None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:08.697430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-7bd4c537-9e24-43af-acec-9a3bac20468b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:08.697430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:08.697430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:08.697430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:08.697430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:08.697430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:08.697430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:08.697430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:08.697430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:08.697430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:08.697430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:08.697430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:08.697430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:08.697430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:08.697430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:08.698526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:08.698526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:08.698526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:08.698526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:08.698526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:08.698526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:08.698526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:08.698526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:08.698526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:08.698526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:08.698526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:08.698526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:08.698526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:08.698526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:08.698526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:08.698526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:08.698526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:08.699646 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:08.699646 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:08.699646 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:08.699646 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:08.699646 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 328/653] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:08.699646 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4c537-9e24-43af-acec-9a3bac20468b None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:08.810161 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ee0a8-f605-409f-b389-a45a4e4a3fbd None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:08.833006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4c537-9e24-43af-acec-9a3bac20468b None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:08.840382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:08.854707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e83b11-abce-4d17-bb62-55fe335699c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:08.855794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e83b11-abce-4d17-bb62-55fe335699c3 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:08.856246 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e83b11-abce-4d17-bb62-55fe335699c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:08.856686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e83b11-abce-4d17-bb62-55fe335699c3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:08.865404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e83b11-abce-4d17-bb62-55fe335699c3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:08.867229 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e83b11-abce-4d17-bb62-55fe335699c3 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:08.868830 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6f435-dc05-4c17-9a8a-272048e69b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:08.869563 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6f435-dc05-4c17-9a8a-272048e69b38 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:08.869900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6f435-dc05-4c17-9a8a-272048e69b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:08.870246 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6f435-dc05-4c17-9a8a-272048e69b38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:08.884553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-09e83b11-abce-4d17-bb62-55fe335699c3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:53:08.884553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:08.884553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:08.884553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:08.884553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:08.884553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:08.884553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:08.884553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:08.884553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:08.884553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:08.884553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:08.884553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:08.884553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:08.884553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:08.884553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:08.884553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:08.885737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:08.886872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:08.886872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:08.886872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:08.886872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:08.886872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:08.886872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:08.886872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:08.886872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:08.886872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:53:08.886872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:08.889458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 326/654] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Jul 12 21:53:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:08.890666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e83b11-abce-4d17-bb62-55fe335699c3 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:08.902535 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:08.946555 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6f435-dc05-4c17-9a8a-272048e69b38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:08.948189 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6f435-dc05-4c17-9a8a-272048e69b38 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:09.012304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e83b11-abce-4d17-bb62-55fe335699c3 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:09.051760 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64888f79-d35b-4fea-8431-f95427eede4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:09.052433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64888f79-d35b-4fea-8431-f95427eede4a None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:09.052735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64888f79-d35b-4fea-8431-f95427eede4a None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:09.053028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64888f79-d35b-4fea-8431-f95427eede4a None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:09.053755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64888f79-d35b-4fea-8431-f95427eede4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:09.054067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64888f79-d35b-4fea-8431-f95427eede4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:09.056111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64888f79-d35b-4fea-8431-f95427eede4a None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:09.056347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 329/655] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:53:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:53:09.057409 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6f435-dc05-4c17-9a8a-272048e69b38 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:09.066092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:09.078348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 327/656] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:09.079501 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64888f79-d35b-4fea-8431-f95427eede4a None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:09.155884 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:09.168522 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6f435-dc05-4c17-9a8a-272048e69b38 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:09.195590 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64888f79-d35b-4fea-8431-f95427eede4a None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:09.204165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227f10e-d45e-480e-a198-e637d1831617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:09.204497 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227f10e-d45e-480e-a198-e637d1831617 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:09.205194 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227f10e-d45e-480e-a198-e637d1831617 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:09.205478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227f10e-d45e-480e-a198-e637d1831617 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:09.206325 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227f10e-d45e-480e-a198-e637d1831617 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:09.206635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227f10e-d45e-480e-a198-e637d1831617 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:09.208279 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227f10e-d45e-480e-a198-e637d1831617 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:09.228579 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 330/657] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Jul 12 21:53:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:09.229480 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227f10e-d45e-480e-a198-e637d1831617 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:09.237229 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b62483-ed3a-4e64-86da-43ec73e5b196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:09.238445 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62483-ed3a-4e64-86da-43ec73e5b196 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:09.238947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62483-ed3a-4e64-86da-43ec73e5b196 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:09.239435 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62483-ed3a-4e64-86da-43ec73e5b196 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:09.251998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b62483-ed3a-4e64-86da-43ec73e5b196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:09.253899 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b62483-ed3a-4e64-86da-43ec73e5b196 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:09.273714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-48b62483-ed3a-4e64-86da-43ec73e5b196 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:09.273714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:09.273714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:09.273714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:09.273714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:09.273714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:09.273714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:09.273714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:09.273714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:09.273714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:09.273714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:09.273714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:09.273714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:09.273714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:09.273714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:09.274662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:09.274662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:09.274662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:09.274662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:09.274662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:09.274662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:09.274662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:09.274662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:09.274662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:09.274662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:09.274662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:09.274662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:09.274662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:09.274662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:09.274662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:09.274662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:09.274662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:09.275556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:09.275556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:09.275556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:09.275556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:09.282389 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 328/658] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:09.283576 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b62483-ed3a-4e64-86da-43ec73e5b196 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:09.295340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:09.337095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227f10e-d45e-480e-a198-e637d1831617 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:09.374824 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2fab3-6b25-4578-b1f1-1d1d72e2ff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:09.375754 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2fab3-6b25-4578-b1f1-1d1d72e2ff7b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:09.376023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2fab3-6b25-4578-b1f1-1d1d72e2ff7b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:09.376304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2fab3-6b25-4578-b1f1-1d1d72e2ff7b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:09.385101 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2fab3-6b25-4578-b1f1-1d1d72e2ff7b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:09.386657 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2fab3-6b25-4578-b1f1-1d1d72e2ff7b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:09.402221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b62483-ed3a-4e64-86da-43ec73e5b196 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:09.403227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-d1d2fab3-6b25-4578-b1f1-1d1d72e2ff7b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 21:53:09.403227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:09.403227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:09.403227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:09.403227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:09.403227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:09.403227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:09.403227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:09.403227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:09.403227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:09.403227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:09.403227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:09.403227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:09.403227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:09.403227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:09.403227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:09.404131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:09.405514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:09.405514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:09.405514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:09.405514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:09.405514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:09.405514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:09.405514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:09.405514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:09.405514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 21:53:09.405514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:09.409222 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 331/659] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:53:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:09.410021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2fab3-6b25-4578-b1f1-1d1d72e2ff7b None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:09.429761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:09.446945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d2ca4-f0b4-49ae-a896-51de10706d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:09.447596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d2ca4-f0b4-49ae-a896-51de10706d5c None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:09.448025 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d2ca4-f0b4-49ae-a896-51de10706d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:09.448406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d2ca4-f0b4-49ae-a896-51de10706d5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:09.526026 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d2ca4-f0b4-49ae-a896-51de10706d5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:09.527918 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d2ca4-f0b4-49ae-a896-51de10706d5c None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:09.533975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2fab3-6b25-4578-b1f1-1d1d72e2ff7b None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:09.560667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:09.572944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f428576-a17c-4bb4-98e6-de977a0d42a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:09.573851 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f428576-a17c-4bb4-98e6-de977a0d42a8 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:09.574363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f428576-a17c-4bb4-98e6-de977a0d42a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:09.574837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f428576-a17c-4bb4-98e6-de977a0d42a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:09.575753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f428576-a17c-4bb4-98e6-de977a0d42a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:09.576328 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f428576-a17c-4bb4-98e6-de977a0d42a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:09.578049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f428576-a17c-4bb4-98e6-de977a0d42a8 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:09.591400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 332/660] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:09.592604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f428576-a17c-4bb4-98e6-de977a0d42a8 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:09.647868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 329/661] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:53:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:53:09.649712 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d2ca4-f0b4-49ae-a896-51de10706d5c None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:09.672698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:09.711836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f428576-a17c-4bb4-98e6-de977a0d42a8 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:09.752399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1c0c2-599a-425b-b212-d74eb13a33bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:09.753108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1c0c2-599a-425b-b212-d74eb13a33bd None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:09.753415 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1c0c2-599a-425b-b212-d74eb13a33bd None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:09.753679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1c0c2-599a-425b-b212-d74eb13a33bd None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:09.754519 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1c0c2-599a-425b-b212-d74eb13a33bd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:09.754834 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1c0c2-599a-425b-b212-d74eb13a33bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:09.756345 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1c0c2-599a-425b-b212-d74eb13a33bd None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:09.757081 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d2ca4-f0b4-49ae-a896-51de10706d5c None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:09.775684 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 333/662] 10.222.0.44 () {56 vars in 1139 bytes} [Tue Jul 12 21:53:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:09.777205 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0653d58-5fea-467e-ad41-6016591524a9 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:09.777439 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0653d58-5fea-467e-ad41-6016591524a9 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:09.777719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0653d58-5fea-467e-ad41-6016591524a9 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:09.780830 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 334/663] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:09.780830 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0653d58-5fea-467e-ad41-6016591524a9 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:09.799532 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7b480-2fe2-46ed-b673-b9fd01759446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:09.800338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7b480-2fe2-46ed-b673-b9fd01759446 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:09.800624 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7b480-2fe2-46ed-b673-b9fd01759446 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:09.800947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7b480-2fe2-46ed-b673-b9fd01759446 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:09.808123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7b480-2fe2-46ed-b673-b9fd01759446 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:09.809640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7b480-2fe2-46ed-b673-b9fd01759446 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:09.823908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-58d7b480-2fe2-46ed-b673-b9fd01759446 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:09.823908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:09.823908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:09.823908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:09.823908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:09.823908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:09.823908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:09.823908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:09.823908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:09.823908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:09.823908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:09.823908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:09.823908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:09.823908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:09.823908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:09.825035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:09.825035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:09.825035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:09.825035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:09.825035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:09.825035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:09.825035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:09.825035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:09.825035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:09.825035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:09.825035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:09.825035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:09.825035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:09.825035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:09.825035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:09.825035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:09.825035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:09.825925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:09.825925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:09.825925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:09.825925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:09.825925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:09.825925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:09.825925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:09.825925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:09.829982 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 330/664] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:09.830858 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7b480-2fe2-46ed-b673-b9fd01759446 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:09.832798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:09.895454 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0653d58-5fea-467e-ad41-6016591524a9 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:09.924140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:09.932368 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4603a-b57c-476d-a3fc-1bca7d789622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:09.933249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4603a-b57c-476d-a3fc-1bca7d789622 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:09.933573 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4603a-b57c-476d-a3fc-1bca7d789622 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:09.933848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4603a-b57c-476d-a3fc-1bca7d789622 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:09.944229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4603a-b57c-476d-a3fc-1bca7d789622 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:09.945811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4603a-b57c-476d-a3fc-1bca7d789622 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:09.947542 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7b480-2fe2-46ed-b673-b9fd01759446 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:09.964751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-6da4603a-b57c-476d-a3fc-1bca7d789622 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:09.964751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:09.964751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:09.964751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:09.964751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:09.964751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:09.964751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:09.964751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:09.964751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:09.964751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:09.964751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:09.964751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:09.964751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:09.964751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:09.964751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:09.965688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:09.965688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:09.965688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:09.965688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:09.965688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:09.965688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:09.965688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:09.965688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:09.965688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:09.965688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:09.965688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:09.965688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:09.965688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:09.965688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:09.965688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:09.965688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:09.965688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:09.967748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:09.967748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:09.967748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:09.967748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:09.967748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 335/665] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:09.967748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3b954-47e8-4190-a818-553af4e7aa4c None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:09.968092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3b954-47e8-4190-a818-553af4e7aa4c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:09.968092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3b954-47e8-4190-a818-553af4e7aa4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:09.985030 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-37f3b954-47e8-4190-a818-553af4e7aa4c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:09.986253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-37f3b954-47e8-4190-a818-553af4e7aa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:09.996584 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c647e-ed4e-4486-ad90-58c963478d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:09.997640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c647e-ed4e-4486-ad90-58c963478d9b None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:09.998074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c647e-ed4e-4486-ad90-58c963478d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:09.998524 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c647e-ed4e-4486-ad90-58c963478d9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:10.060837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 336/666] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:53:10.060837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3b954-47e8-4190-a818-553af4e7aa4c None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:10.086748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c647e-ed4e-4486-ad90-58c963478d9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:10.088412 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c647e-ed4e-4486-ad90-58c963478d9b None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:10.093527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:10.203516 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:10.206808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3b954-47e8-4190-a818-553af4e7aa4c None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:10.234748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 331/667] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:53:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:53:10.235798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c647e-ed4e-4486-ad90-58c963478d9b None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:10.250653 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256be95-ebf4-46b6-a910-44584babd233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:10.251695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256be95-ebf4-46b6-a910-44584babd233 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:10.252245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256be95-ebf4-46b6-a910-44584babd233 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:10.252527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256be95-ebf4-46b6-a910-44584babd233 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:10.253516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256be95-ebf4-46b6-a910-44584babd233 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:10.253837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256be95-ebf4-46b6-a910-44584babd233 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:10.255581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256be95-ebf4-46b6-a910-44584babd233 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:10.272539 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 337/668] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:10.273588 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256be95-ebf4-46b6-a910-44584babd233 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:10.347813 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c647e-ed4e-4486-ad90-58c963478d9b None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:10.374027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:10.387284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d610f80-e03c-4d70-b00b-126b6ee55080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:10.388052 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d610f80-e03c-4d70-b00b-126b6ee55080 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:10.388234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d610f80-e03c-4d70-b00b-126b6ee55080 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:10.388499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d610f80-e03c-4d70-b00b-126b6ee55080 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:10.389251 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d610f80-e03c-4d70-b00b-126b6ee55080 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:10.389590 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d610f80-e03c-4d70-b00b-126b6ee55080 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:10.390916 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d610f80-e03c-4d70-b00b-126b6ee55080 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:10.393080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256be95-ebf4-46b6-a910-44584babd233 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:10.403437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 332/669] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:10.404258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d610f80-e03c-4d70-b00b-126b6ee55080 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:10.438254 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c95b3-48a5-4952-91bf-28d852940c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:10.439046 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c95b3-48a5-4952-91bf-28d852940c40 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:10.439329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c95b3-48a5-4952-91bf-28d852940c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:10.439611 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c95b3-48a5-4952-91bf-28d852940c40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:10.451037 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:10.453448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c95b3-48a5-4952-91bf-28d852940c40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:10.461822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-8f8c95b3-48a5-4952-91bf-28d852940c40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:10.461822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:10.461822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:10.461822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:10.461822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:10.461822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:10.461822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:10.461822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:10.461822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:10.461822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:10.461822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:10.461822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:10.461822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:10.461822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:10.461822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:10.461822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:10.462778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:10.462778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:10.462778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:10.462778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:10.462778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:10.462778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:10.462778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:10.462778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:10.462778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:10.462778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:10.462778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:10.462778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:10.462778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:10.462778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:10.462778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:10.462778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:10.463618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:10.463618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:10.463618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:10.463618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:10.463618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:10.463618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:10.463618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:10.463618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:10.463618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:10.463618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:10.463618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:10.463618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:10.463618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 338/670] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:53:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:10.464322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c95b3-48a5-4952-91bf-28d852940c40 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:10.517489 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d610f80-e03c-4d70-b00b-126b6ee55080 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:10.554238 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68664599-60f6-41ed-809e-4f518c988b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:10.555763 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68664599-60f6-41ed-809e-4f518c988b78 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:10.556049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68664599-60f6-41ed-809e-4f518c988b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:10.556328 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68664599-60f6-41ed-809e-4f518c988b78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:10.567182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68664599-60f6-41ed-809e-4f518c988b78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:10.568661 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68664599-60f6-41ed-809e-4f518c988b78 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:10.583916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c95b3-48a5-4952-91bf-28d852940c40 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:10.587117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-68664599-60f6-41ed-809e-4f518c988b78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:10.587117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:10.587117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:10.587117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:10.587117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:10.587117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:10.587117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:10.587117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:10.587117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:10.587117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:10.587117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:10.587117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:10.587117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:10.587117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:10.587117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:10.588100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:10.588100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:10.588100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:10.588100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:10.588100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:10.588100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:10.588100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:10.588100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:10.588100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:10.588100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:10.588100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:10.588100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:10.588100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:10.588100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:10.588100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:10.588100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:10.588100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:10.589316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:10.589316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:10.589316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:10.589316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:10.589316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 333/671] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:10.589586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35545dca-9092-4dee-b565-e7ff6948cf1d None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:10.589855 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35545dca-9092-4dee-b565-e7ff6948cf1d None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:10.590135 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35545dca-9092-4dee-b565-e7ff6948cf1d None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:10.591398 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 334/672] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:10.592091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35545dca-9092-4dee-b565-e7ff6948cf1d None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:10.621263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9241472-d9ec-415c-8727-d599a6eaf5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:10.621838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9241472-d9ec-415c-8727-d599a6eaf5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:10.622121 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9241472-d9ec-415c-8727-d599a6eaf5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:10.622398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9241472-d9ec-415c-8727-d599a6eaf5e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:10.623227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9241472-d9ec-415c-8727-d599a6eaf5e2 None admin] RBAC: Adding query filter params (role.id=f1e42ea5a1c440bba915b459d2744858, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=c5c9ed6f9fd542069cf914003b636c1c) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:10.623538 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9241472-d9ec-415c-8727-d599a6eaf5e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:10.624995 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9241472-d9ec-415c-8727-d599a6eaf5e2 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:10.631483 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 339/673] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:53:10 2022] GET /identity/v3/role_assignments?user.id=c5c9ed6f9fd542069cf914003b636c1c&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=f1e42ea5a1c440bba915b459d2744858 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:10.632292 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9241472-d9ec-415c-8727-d599a6eaf5e2 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:10.644555 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:10.698886 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:10.714685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35545dca-9092-4dee-b565-e7ff6948cf1d None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:10.752793 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9241472-d9ec-415c-8727-d599a6eaf5e2 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:10.758825 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99f789-59d7-485f-ac4d-5b3dd7f76c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:10.759667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99f789-59d7-485f-ac4d-5b3dd7f76c52 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:10.759952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99f789-59d7-485f-ac4d-5b3dd7f76c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:10.760233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99f789-59d7-485f-ac4d-5b3dd7f76c52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:10.768025 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee99f789-59d7-485f-ac4d-5b3dd7f76c52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:10.769562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee99f789-59d7-485f-ac4d-5b3dd7f76c52 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:10.784486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-ee99f789-59d7-485f-ac4d-5b3dd7f76c52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:10.784486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:10.784486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:10.784486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:10.784486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:10.784486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:10.784486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:10.784486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:10.784486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:10.784486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:10.784486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:10.784486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:10.784486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:10.784486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:10.784486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:10.785490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:10.785490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:10.785490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:10.785490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:10.785490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:10.785490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:10.785490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:10.785490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:10.785490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:10.785490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:10.785490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:10.785490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:10.785490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:10.785490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:10.785490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:10.785490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:10.785490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:10.786748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:10.786748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:10.786748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:10.786748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:10.786748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:10.786748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:10.786748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:10.786748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:10.786748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 335/674] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:10.786748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99f789-59d7-485f-ac4d-5b3dd7f76c52 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:10.800798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4532ba-d010-44c6-90b4-5e35a7fb5472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:10.800798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4532ba-d010-44c6-90b4-5e35a7fb5472 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:10.800798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4532ba-d010-44c6-90b4-5e35a7fb5472 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:10.801834 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4532ba-d010-44c6-90b4-5e35a7fb5472 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:10.804829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4532ba-d010-44c6-90b4-5e35a7fb5472 None admin] RBAC: Adding query filter params (name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:10.804829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4532ba-d010-44c6-90b4-5e35a7fb5472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:10.804829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4532ba-d010-44c6-90b4-5e35a7fb5472 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:10.813347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 340/675] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:53:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:10.814826 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a1140-4ab3-40ca-aeaf-acc9848a6854 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:10.815136 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a1140-4ab3-40ca-aeaf-acc9848a6854 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:10.815416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a1140-4ab3-40ca-aeaf-acc9848a6854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:10.833624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-ef6a1140-4ab3-40ca-aeaf-acc9848a6854 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:10.834817 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ef6a1140-4ab3-40ca-aeaf-acc9848a6854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:10.905072 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99f789-59d7-485f-ac4d-5b3dd7f76c52 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:10.905318 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:10.910388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 341/676] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:53:10.916826 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a1140-4ab3-40ca-aeaf-acc9848a6854 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:10.951957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba27d3-c8d9-4fd7-9f9a-4880ae126959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:10.952619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba27d3-c8d9-4fd7-9f9a-4880ae126959 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:10.952930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba27d3-c8d9-4fd7-9f9a-4880ae126959 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:10.953204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba27d3-c8d9-4fd7-9f9a-4880ae126959 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:10.953928 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba27d3-c8d9-4fd7-9f9a-4880ae126959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:10.954235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba27d3-c8d9-4fd7-9f9a-4880ae126959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:10.955608 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba27d3-c8d9-4fd7-9f9a-4880ae126959 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:10.971191 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 336/677] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:10.971910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba27d3-c8d9-4fd7-9f9a-4880ae126959 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:10.987842 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:11.034602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a1140-4ab3-40ca-aeaf-acc9848a6854 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:11.072390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f77730-785c-47fa-a0c6-e700b29c1134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:11.073197 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f77730-785c-47fa-a0c6-e700b29c1134 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:11.073487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f77730-785c-47fa-a0c6-e700b29c1134 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:11.073878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f77730-785c-47fa-a0c6-e700b29c1134 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:11.074691 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f77730-785c-47fa-a0c6-e700b29c1134 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:11.075001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f77730-785c-47fa-a0c6-e700b29c1134 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:11.076442 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f77730-785c-47fa-a0c6-e700b29c1134 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:11.087485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba27d3-c8d9-4fd7-9f9a-4880ae126959 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:11.093969 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 342/678] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:11.093969 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f77730-785c-47fa-a0c6-e700b29c1134 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:11.131428 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21489f33-52c5-45f4-812c-4ca3a41d00cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:11.132218 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21489f33-52c5-45f4-812c-4ca3a41d00cf None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:11.133112 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21489f33-52c5-45f4-812c-4ca3a41d00cf None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:11.134253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21489f33-52c5-45f4-812c-4ca3a41d00cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:11.145395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21489f33-52c5-45f4-812c-4ca3a41d00cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:11.146905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21489f33-52c5-45f4-812c-4ca3a41d00cf None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:11.154395 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:11.168483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-21489f33-52c5-45f4-812c-4ca3a41d00cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:11.168483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:11.168483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:11.168483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:11.168483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:11.168483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:11.168483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:11.168483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:11.168483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:11.168483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:11.168483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:11.168483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:11.168483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:11.168483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:11.168483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:11.169426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:11.169426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:11.169426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:11.169426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:11.169426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:11.169426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:11.169426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:11.169426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:11.169426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:11.169426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:11.169426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:11.169426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:11.169426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:11.169426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:11.169426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:11.169426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:11.169426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:11.170316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:11.170316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:11.170316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:11.170316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:11.170316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 337/679] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:11.171060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21489f33-52c5-45f4-812c-4ca3a41d00cf None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:11.205232 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f77730-785c-47fa-a0c6-e700b29c1134 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:11.238322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:11.242174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beaf351-d4c9-4429-81e5-b6d586d6e75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:11.242951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beaf351-d4c9-4429-81e5-b6d586d6e75c None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:11.243570 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beaf351-d4c9-4429-81e5-b6d586d6e75c None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:11.243570 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beaf351-d4c9-4429-81e5-b6d586d6e75c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:11.244144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beaf351-d4c9-4429-81e5-b6d586d6e75c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:11.245566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beaf351-d4c9-4429-81e5-b6d586d6e75c None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:11.259802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-1beaf351-d4c9-4429-81e5-b6d586d6e75c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 21:53:11.259802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:11.259802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:11.259802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:11.259802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:11.259802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:11.259802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:11.259802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:11.259802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 21:53:11.259802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:11.259802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:11.259802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:11.259802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:11.259802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:11.259802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:11.259802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:11.260832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:11.260832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:11.260832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:11.260832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:11.260832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:11.260832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:11.260832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:11.260832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:11.260832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:11.260832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:11.260832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:11.260832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:11.260832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:11.260832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:11.260832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:11.260832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:11.261665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:11.261665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:11.261665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:11.261665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:11.261665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:11.261665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:11.261665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:11.261665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:11.261665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:11.261665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:11.261665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:11.261665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:11.261665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:11.261665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 21:53:11.261665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:11.261665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 343/680] 10.222.0.44 () {56 vars in 1129 bytes} [Tue Jul 12 21:53:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:11.262509 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beaf351-d4c9-4429-81e5-b6d586d6e75c None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:11.285642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21489f33-52c5-45f4-812c-4ca3a41d00cf None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:11.322837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90e20b-1fdb-4191-8f40-300a040abed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:11.323632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90e20b-1fdb-4191-8f40-300a040abed1 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:11.323915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90e20b-1fdb-4191-8f40-300a040abed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:11.324190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90e20b-1fdb-4191-8f40-300a040abed1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:11.331325 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90e20b-1fdb-4191-8f40-300a040abed1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:11.332805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90e20b-1fdb-4191-8f40-300a040abed1 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:11.347844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-9e90e20b-1fdb-4191-8f40-300a040abed1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:11.347844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:11.347844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:11.347844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:11.347844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:11.347844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:11.347844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:11.347844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:11.347844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:11.347844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:11.347844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:11.347844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:11.347844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:11.347844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:11.347844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:11.348981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:11.348981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:11.348981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:11.348981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:11.348981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:11.348981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:11.348981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:11.348981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:11.348981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:11.348981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:11.348981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:11.348981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:11.348981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:11.348981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:11.348981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:11.348981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:11.348981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:11.349950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:11.349950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:11.349950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:11.349950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:11.349950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:11.349950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:11.349950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:11.349950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:11.349950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 338/681] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:11.349950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90e20b-1fdb-4191-8f40-300a040abed1 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:11.383733 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beaf351-d4c9-4429-81e5-b6d586d6e75c None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:11.409825 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:11.422261 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdb14b-07f4-44de-b06e-a3018b913314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:11.422951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdb14b-07f4-44de-b06e-a3018b913314 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:11.423148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdb14b-07f4-44de-b06e-a3018b913314 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:11.423401 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdb14b-07f4-44de-b06e-a3018b913314 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:11.424243 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdb14b-07f4-44de-b06e-a3018b913314 None admin] RBAC: Adding query filter params (role.id=f1e42ea5a1c440bba915b459d2744858, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=24915625354f418eb7bc4d5ada85fdb0) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:11.424884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdb14b-07f4-44de-b06e-a3018b913314 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:11.426027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdb14b-07f4-44de-b06e-a3018b913314 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:11.432448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 344/682] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:53:11 2022] GET /identity/v3/role_assignments?user.id=24915625354f418eb7bc4d5ada85fdb0&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=f1e42ea5a1c440bba915b459d2744858 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:11.438478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdb14b-07f4-44de-b06e-a3018b913314 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:11.463557 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90e20b-1fdb-4191-8f40-300a040abed1 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:11.490841 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:11.505373 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3dfba-310d-4e73-a3bf-2746ee4af2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:11.506059 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3dfba-310d-4e73-a3bf-2746ee4af2fc None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:11.506349 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3dfba-310d-4e73-a3bf-2746ee4af2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:11.506660 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3dfba-310d-4e73-a3bf-2746ee4af2fc None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:11.507351 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3dfba-310d-4e73-a3bf-2746ee4af2fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:11.508242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3dfba-310d-4e73-a3bf-2746ee4af2fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:11.510197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3dfba-310d-4e73-a3bf-2746ee4af2fc None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:11.523718 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 339/683] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:11.524625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3dfba-310d-4e73-a3bf-2746ee4af2fc None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:11.569432 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdb14b-07f4-44de-b06e-a3018b913314 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:11.610251 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef4516-e789-4d32-895d-b04fe25ce9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:11.611726 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef4516-e789-4d32-895d-b04fe25ce9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:11.612017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef4516-e789-4d32-895d-b04fe25ce9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:11.612300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef4516-e789-4d32-895d-b04fe25ce9a4 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:11.613082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fef4516-e789-4d32-895d-b04fe25ce9a4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:11.613392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fef4516-e789-4d32-895d-b04fe25ce9a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:11.614785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fef4516-e789-4d32-895d-b04fe25ce9a4 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:11.620898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 345/684] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Jul 12 21:53:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:11.621670 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef4516-e789-4d32-895d-b04fe25ce9a4 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:11.643016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3dfba-310d-4e73-a3bf-2746ee4af2fc None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:11.692004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7941f-e0c4-4d93-b5c0-fe3a10bd846e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:11.693374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:11.694328 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7941f-e0c4-4d93-b5c0-fe3a10bd846e None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:11.694798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7941f-e0c4-4d93-b5c0-fe3a10bd846e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:11.695814 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7941f-e0c4-4d93-b5c0-fe3a10bd846e None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:11.696999 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7941f-e0c4-4d93-b5c0-fe3a10bd846e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:11.698116 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7941f-e0c4-4d93-b5c0-fe3a10bd846e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:11.699778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7941f-e0c4-4d93-b5c0-fe3a10bd846e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:11.716397 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 340/685] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:11.717456 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7941f-e0c4-4d93-b5c0-fe3a10bd846e None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:11.757025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef4516-e789-4d32-895d-b04fe25ce9a4 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:11.790408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:11.807788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa033ced-a2fb-4c3f-8d9d-415c17981a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:11.808648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa033ced-a2fb-4c3f-8d9d-415c17981a98 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:11.808968 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa033ced-a2fb-4c3f-8d9d-415c17981a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:11.809245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa033ced-a2fb-4c3f-8d9d-415c17981a98 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:11.818655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa033ced-a2fb-4c3f-8d9d-415c17981a98 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:11.820740 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa033ced-a2fb-4c3f-8d9d-415c17981a98 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:11.837412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-aa033ced-a2fb-4c3f-8d9d-415c17981a98 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 21:53:11.837412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:11.837412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:11.837412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:11.837412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:11.837412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:11.837412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:11.837412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:11.837412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:11.837412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:11.837412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:11.837412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:11.837412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:11.837412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:11.837412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:11.837412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:11.838425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:11.839358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:11.839358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:11.839358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:11.839358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:11.839358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:11.839358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:11.839358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:11.839358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:11.839358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 21:53:11.839358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:11.844745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7941f-e0c4-4d93-b5c0-fe3a10bd846e None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:11.844940 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 346/686] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:53:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:11.844940 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa033ced-a2fb-4c3f-8d9d-415c17981a98 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:11.889883 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7bd22-26d9-4114-bea7-ca1ba1f847f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:11.890606 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7bd22-26d9-4114-bea7-ca1ba1f847f3 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:11.890897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7bd22-26d9-4114-bea7-ca1ba1f847f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:11.891176 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7bd22-26d9-4114-bea7-ca1ba1f847f3 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:11.891892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7bd22-26d9-4114-bea7-ca1ba1f847f3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:11.892201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7bd22-26d9-4114-bea7-ca1ba1f847f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:11.894187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7bd22-26d9-4114-bea7-ca1ba1f847f3 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:11.903381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 341/687] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Jul 12 21:53:11 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:11.904472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7bd22-26d9-4114-bea7-ca1ba1f847f3 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:11.980304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa033ced-a2fb-4c3f-8d9d-415c17981a98 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:11.996180 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:12.024949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75285c6a-da28-4c26-b023-eca73196eff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:12.026051 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75285c6a-da28-4c26-b023-eca73196eff8 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:12.026599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75285c6a-da28-4c26-b023-eca73196eff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:12.027112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75285c6a-da28-4c26-b023-eca73196eff8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:12.028275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75285c6a-da28-4c26-b023-eca73196eff8 None admin] RBAC: Adding query filter params (role.id=f1e42ea5a1c440bba915b459d2744858, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=53b891a734a64b5983564e207de0359d) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:12.028936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75285c6a-da28-4c26-b023-eca73196eff8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:12.030669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75285c6a-da28-4c26-b023-eca73196eff8 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:12.038091 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 347/688] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:53:11 2022] GET /identity/v3/role_assignments?user.id=53b891a734a64b5983564e207de0359d&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=f1e42ea5a1c440bba915b459d2744858 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:12.038643 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7bd22-26d9-4114-bea7-ca1ba1f847f3 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:12.050343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:12.098020 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01a3a8-39ef-430a-9678-dbd6be29cd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:12.099377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01a3a8-39ef-430a-9678-dbd6be29cd7b None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:12.100054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01a3a8-39ef-430a-9678-dbd6be29cd7b None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:12.100597 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01a3a8-39ef-430a-9678-dbd6be29cd7b None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:12.102310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01a3a8-39ef-430a-9678-dbd6be29cd7b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:12.102865 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01a3a8-39ef-430a-9678-dbd6be29cd7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:12.104546 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01a3a8-39ef-430a-9678-dbd6be29cd7b None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:12.129673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 342/689] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Jul 12 21:53:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:12.135440 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75285c6a-da28-4c26-b023-eca73196eff8 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:12.255886 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75285c6a-da28-4c26-b023-eca73196eff8 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:12.297782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ff3fe-44ab-4d7b-992a-ec222ee65a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:12.298438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ff3fe-44ab-4d7b-992a-ec222ee65a67 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:12.298720 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ff3fe-44ab-4d7b-992a-ec222ee65a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:12.298996 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ff3fe-44ab-4d7b-992a-ec222ee65a67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:12.348927 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:12.375403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ff3fe-44ab-4d7b-992a-ec222ee65a67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:12.376916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ff3fe-44ab-4d7b-992a-ec222ee65a67 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:12.485731 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 348/690] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:53:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:53:12.491282 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01a3a8-39ef-430a-9678-dbd6be29cd7b None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:12.597085 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01a3a8-39ef-430a-9678-dbd6be29cd7b None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:12.647283 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762c116-2dec-435a-b425-a8bbc15d79f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:12.648503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762c116-2dec-435a-b425-a8bbc15d79f4 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:12.649043 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762c116-2dec-435a-b425-a8bbc15d79f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:12.649569 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762c116-2dec-435a-b425-a8bbc15d79f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:12.663283 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762c116-2dec-435a-b425-a8bbc15d79f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:12.665612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762c116-2dec-435a-b425-a8bbc15d79f4 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:12.693258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-2762c116-2dec-435a-b425-a8bbc15d79f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:12.693258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:12.693258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:12.693258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:12.693258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:12.693258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:12.693258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:12.693258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:12.693258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:12.693258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:12.693258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:12.693258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:12.693258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:12.693258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:12.693258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:12.694216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:12.694216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:12.694216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:12.694216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:12.694216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:12.694216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:12.694216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:12.694216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:12.694216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:12.694216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:12.694216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:12.694216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:12.694216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:12.694216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:12.694216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:12.694216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:12.694216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:12.695686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:12.695686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:12.695686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:12.695686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:12.701908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 343/691] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:12.703879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ff3fe-44ab-4d7b-992a-ec222ee65a67 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:12.846647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:12.859247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413986d-33d1-4cec-b7fb-4f3eecc707ec None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:12.859533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413986d-33d1-4cec-b7fb-4f3eecc707ec None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:12.859736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413986d-33d1-4cec-b7fb-4f3eecc707ec None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:12.861473 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 344/692] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:53:12.866634 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763eac8-5bf0-4795-a8fb-875f489f2779 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:12.867733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763eac8-5bf0-4795-a8fb-875f489f2779 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:12.868277 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763eac8-5bf0-4795-a8fb-875f489f2779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:12.883144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ff3fe-44ab-4d7b-992a-ec222ee65a67 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:12.910276 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-a763eac8-5bf0-4795-a8fb-875f489f2779 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:12.911998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a763eac8-5bf0-4795-a8fb-875f489f2779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:12.945521 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25877f-ad9a-46ee-980d-16652597a48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:12.946570 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25877f-ad9a-46ee-980d-16652597a48a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:12.947082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25877f-ad9a-46ee-980d-16652597a48a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:12.947585 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25877f-ad9a-46ee-980d-16652597a48a None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:12.947726 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:12.948880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25877f-ad9a-46ee-980d-16652597a48a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:12.948880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25877f-ad9a-46ee-980d-16652597a48a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:12.950318 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25877f-ad9a-46ee-980d-16652597a48a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:12.975766 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 349/693] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:12.980244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25877f-ad9a-46ee-980d-16652597a48a None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:13.053335 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 345/694] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:53:13.079479 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763eac8-5bf0-4795-a8fb-875f489f2779 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:13.160403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25877f-ad9a-46ee-980d-16652597a48a None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:13.208952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d254dca-429b-4b46-8329-4896e1bdccdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:13.209810 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d254dca-429b-4b46-8329-4896e1bdccdb None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:13.210089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d254dca-429b-4b46-8329-4896e1bdccdb None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:13.210357 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d254dca-429b-4b46-8329-4896e1bdccdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:13.218554 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:13.227851 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d254dca-429b-4b46-8329-4896e1bdccdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:13.229355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d254dca-429b-4b46-8329-4896e1bdccdb None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:13.260728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-5d254dca-429b-4b46-8329-4896e1bdccdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:13.260728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:13.260728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:13.260728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:13.260728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:13.260728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:13.260728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:13.260728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:13.260728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:13.260728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:13.260728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:13.260728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:13.260728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:13.260728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:13.260728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:13.262633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:13.262633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:13.262633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:13.262633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:13.262633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:13.262633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:13.262633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:13.262633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:13.262633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:13.262633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:13.262633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:13.262633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:13.262633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:13.262633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:13.262633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:13.262633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:13.262633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:13.264296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:13.264296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:13.264296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:13.264296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:13.264296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:13.264296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:13.264296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:13.264296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:13.264296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 350/695] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:13.268992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763eac8-5bf0-4795-a8fb-875f489f2779 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:13.269716 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d254dca-429b-4b46-8329-4896e1bdccdb None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:13.310053 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4645f76-ffd4-4e78-a3ed-36e1a54de698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:13.310895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4645f76-ffd4-4e78-a3ed-36e1a54de698 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:13.311180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4645f76-ffd4-4e78-a3ed-36e1a54de698 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:13.311454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4645f76-ffd4-4e78-a3ed-36e1a54de698 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:13.323587 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4645f76-ffd4-4e78-a3ed-36e1a54de698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:13.325148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4645f76-ffd4-4e78-a3ed-36e1a54de698 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:13.335229 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:13.347594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-c4645f76-ffd4-4e78-a3ed-36e1a54de698 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:13.347594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:13.347594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:13.347594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:13.347594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:13.347594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:13.347594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:13.347594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:13.347594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:13.347594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:13.347594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:13.347594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:13.347594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:13.347594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:13.347594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:13.349686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:13.349686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:13.349686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:13.349686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:13.349686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:13.349686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:13.349686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:13.349686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:13.349686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:13.349686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:13.349686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:13.349686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:13.349686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:13.349686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:13.349686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:13.349686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:13.349686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:13.350905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:13.350905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:13.350905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:13.350905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:13.350905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 346/696] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:13.355113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4645f76-ffd4-4e78-a3ed-36e1a54de698 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:13.475401 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4645f76-ffd4-4e78-a3ed-36e1a54de698 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:13.516427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd876c-cc39-4e60-b2d6-a9839a1848ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:13.517233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd876c-cc39-4e60-b2d6-a9839a1848ac None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:13.520731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd876c-cc39-4e60-b2d6-a9839a1848ac None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:13.520731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd876c-cc39-4e60-b2d6-a9839a1848ac None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:13.520731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd876c-cc39-4e60-b2d6-a9839a1848ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:13.520731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd876c-cc39-4e60-b2d6-a9839a1848ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:13.520731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd876c-cc39-4e60-b2d6-a9839a1848ac None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:13.535029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 347/697] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:13.540167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd876c-cc39-4e60-b2d6-a9839a1848ac None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:13.556369 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d254dca-429b-4b46-8329-4896e1bdccdb None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:13.645427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13c61f-12a4-49fe-b179-8fb9daf8ef2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:13.646529 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13c61f-12a4-49fe-b179-8fb9daf8ef2d None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:13.647020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13c61f-12a4-49fe-b179-8fb9daf8ef2d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:13.647517 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13c61f-12a4-49fe-b179-8fb9daf8ef2d None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:13.649220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13c61f-12a4-49fe-b179-8fb9daf8ef2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:13.649417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:13.650140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13c61f-12a4-49fe-b179-8fb9daf8ef2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:13.651403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13c61f-12a4-49fe-b179-8fb9daf8ef2d None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:13.673292 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 351/698] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:13.682050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13c61f-12a4-49fe-b179-8fb9daf8ef2d None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:13.732019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd876c-cc39-4e60-b2d6-a9839a1848ac None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:13.789139 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2309415-1033-4cfa-875d-40f3780ee3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:13.789794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2309415-1033-4cfa-875d-40f3780ee3a6 None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:13.790090 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2309415-1033-4cfa-875d-40f3780ee3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:13.790362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2309415-1033-4cfa-875d-40f3780ee3a6 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:13.791080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2309415-1033-4cfa-875d-40f3780ee3a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:13.792533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2309415-1033-4cfa-875d-40f3780ee3a6 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:13.797886 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:13.833641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 348/699] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:53:13 2022] POST /identity/v3/users => generated 254 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:53:13.876685 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13c61f-12a4-49fe-b179-8fb9daf8ef2d None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:13.920356 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79434512-ebb2-4316-9162-a76eae49046f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:13.921081 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79434512-ebb2-4316-9162-a76eae49046f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:13.921361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79434512-ebb2-4316-9162-a76eae49046f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:13.921635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79434512-ebb2-4316-9162-a76eae49046f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:13.922488 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79434512-ebb2-4316-9162-a76eae49046f None admin] RBAC: Adding query filter params (role.id=f1e42ea5a1c440bba915b459d2744858, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=837bb5ac18554dfdbbcc776689639494) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:13.922803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79434512-ebb2-4316-9162-a76eae49046f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:13.924286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79434512-ebb2-4316-9162-a76eae49046f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:13.934401 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 352/700] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:53:13 2022] GET /identity/v3/role_assignments?user.id=837bb5ac18554dfdbbcc776689639494&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=f1e42ea5a1c440bba915b459d2744858 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:14.141491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ae811-d730-4eac-af5f-c18324562b27 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:14.141827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ae811-d730-4eac-af5f-c18324562b27 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:14.142061 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ae811-d730-4eac-af5f-c18324562b27 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:14.143818 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 349/701] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:14.150260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfd894-c2fe-4817-af65-442ab1a40ec8 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:14.150567 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfd894-c2fe-4817-af65-442ab1a40ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:14.150833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfd894-c2fe-4817-af65-442ab1a40ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:14.172571 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-82cfd894-c2fe-4817-af65-442ab1a40ec8 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:14.173831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-82cfd894-c2fe-4817-af65-442ab1a40ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:14.254609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 353/702] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:53:14.278791 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ae811-d730-4eac-af5f-c18324562b27 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:14.424189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ae811-d730-4eac-af5f-c18324562b27 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:14.435016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd774ea-db38-41ab-9039-8cf3958e672b None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:14.435268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd774ea-db38-41ab-9039-8cf3958e672b None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:14.435556 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd774ea-db38-41ab-9039-8cf3958e672b None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:14.441310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 354/703] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:14.443765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:14.450235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b970a15-a9c0-4912-b27a-80fd0210bc8c None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:14.450514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b970a15-a9c0-4912-b27a-80fd0210bc8c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:14.450801 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b970a15-a9c0-4912-b27a-80fd0210bc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:14.479540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0c217-97c4-4b8a-90e9-45f2a970ff0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:14.480408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0c217-97c4-4b8a-90e9-45f2a970ff0d None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:14.480757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0c217-97c4-4b8a-90e9-45f2a970ff0d None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:14.481038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0c217-97c4-4b8a-90e9-45f2a970ff0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:14.484838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-0b970a15-a9c0-4912-b27a-80fd0210bc8c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:14.485981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0b970a15-a9c0-4912-b27a-80fd0210bc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:14.504639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0c217-97c4-4b8a-90e9-45f2a970ff0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:14.514630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-46f0c217-97c4-4b8a-90e9-45f2a970ff0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:53:14.514630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:14.514630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:14.514630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:14.514630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:14.514630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:14.514630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:14.514630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:14.514630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:14.514630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:14.514630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:14.514630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:14.514630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:14.514630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:14.514630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:14.514630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:14.515607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:14.515607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:14.515607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:14.515607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:14.515607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:14.515607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:14.515607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:14.515607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:14.515607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:14.515607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:14.515607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:14.515607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:14.515607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:14.515607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:14.515607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:14.515607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:14.517332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:14.517332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:14.517332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:14.517332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:14.517332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:14.517332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:14.517332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:14.517332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:14.517332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:14.517332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:14.517332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:53:14.517332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:14.517332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 350/704] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:53:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:14.525395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0c217-97c4-4b8a-90e9-45f2a970ff0d None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:14.565374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:14.589478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 355/705] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:53:14.615256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b970a15-a9c0-4912-b27a-80fd0210bc8c None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:14.677645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0c217-97c4-4b8a-90e9-45f2a970ff0d None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:14.738079 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23372644-3fa0-4d8a-884e-30cf4622e75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:14.738766 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23372644-3fa0-4d8a-884e-30cf4622e75a None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:14.739050 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23372644-3fa0-4d8a-884e-30cf4622e75a None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:14.739329 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23372644-3fa0-4d8a-884e-30cf4622e75a None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:14.740101 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23372644-3fa0-4d8a-884e-30cf4622e75a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:14.740413 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23372644-3fa0-4d8a-884e-30cf4622e75a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:14.741914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23372644-3fa0-4d8a-884e-30cf4622e75a None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:14.766757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 351/706] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:53:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:14.772251 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23372644-3fa0-4d8a-884e-30cf4622e75a None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:14.782904 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b970a15-a9c0-4912-b27a-80fd0210bc8c None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:14.840689 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:14.843362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800877dc-773a-477e-ad2c-e0be0eae552e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:14.844261 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800877dc-773a-477e-ad2c-e0be0eae552e None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:14.844549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800877dc-773a-477e-ad2c-e0be0eae552e None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:14.844858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800877dc-773a-477e-ad2c-e0be0eae552e None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:14.845586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800877dc-773a-477e-ad2c-e0be0eae552e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:14.846955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800877dc-773a-477e-ad2c-e0be0eae552e None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:14.863294 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 356/707] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Jul 12 21:53:14 2022] POST /identity/v3/services => generated 251 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 21:53:14.865056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb00abd-d580-4b21-9fdf-a4e1e3b5e635 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:14.865455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb00abd-d580-4b21-9fdf-a4e1e3b5e635 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:14.865840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb00abd-d580-4b21-9fdf-a4e1e3b5e635 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:14.871788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 357/708] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:14.873473 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c75d3-78e2-4077-b7e3-93340112a2ab None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:14.873891 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c75d3-78e2-4077-b7e3-93340112a2ab None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:14.874239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c75d3-78e2-4077-b7e3-93340112a2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:14.893908 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-476c75d3-78e2-4077-b7e3-93340112a2ab None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:14.895331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-476c75d3-78e2-4077-b7e3-93340112a2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:14.935211 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:14.990451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 358/709] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:14 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:53:14.998956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23372644-3fa0-4d8a-884e-30cf4622e75a None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:15.013250 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c75d3-78e2-4077-b7e3-93340112a2ab None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:15.077520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428df67-585b-47b4-b796-7a4be8ff46a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:15.078424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428df67-585b-47b4-b796-7a4be8ff46a3 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:15.078717 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428df67-585b-47b4-b796-7a4be8ff46a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:15.079001 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428df67-585b-47b4-b796-7a4be8ff46a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:15.094636 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428df67-585b-47b4-b796-7a4be8ff46a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:15.096118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428df67-585b-47b4-b796-7a4be8ff46a3 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:15.122207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-3428df67-585b-47b4-b796-7a4be8ff46a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:15.122207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:15.122207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:15.122207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:15.122207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:15.122207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:15.122207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:15.122207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:15.122207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:15.122207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:15.122207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:15.122207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:15.122207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:15.122207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:15.122207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:15.123172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:15.123172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:15.123172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:15.123172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:15.123172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:15.123172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:15.123172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:15.123172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:15.123172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:15.123172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:15.123172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:15.123172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:15.123172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:15.123172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:15.123172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:15.123172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:15.123172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:15.124049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:15.124049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:15.124049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:15.124049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:15.124049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 352/710] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:15.129861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428df67-585b-47b4-b796-7a4be8ff46a3 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:15.130831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c75d3-78e2-4077-b7e3-93340112a2ab None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:15.170462 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c17fee-460d-47ff-ac99-3daf93d2b2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:15.171263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c17fee-460d-47ff-ac99-3daf93d2b2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:15.171543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c17fee-460d-47ff-ac99-3daf93d2b2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:15.171818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c17fee-460d-47ff-ac99-3daf93d2b2c0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:15.172474 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c17fee-460d-47ff-ac99-3daf93d2b2c0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:15.174950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c17fee-460d-47ff-ac99-3daf93d2b2c0 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:15.188898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-24c17fee-460d-47ff-ac99-3daf93d2b2c0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 21:53:15.188898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:15.188898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:15.188898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:15.188898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:15.188898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:15.188898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:15.188898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:15.188898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 21:53:15.188898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:15.188898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:15.188898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:15.188898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:15.188898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:15.188898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:15.188898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:15.188898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:15.189889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:15.189889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:15.189889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:15.189889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:15.189889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:15.189889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:15.189889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:15.189889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:15.189889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:15.189889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:15.189889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:15.189889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:15.189889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:15.189889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:15.189889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:15.189889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:15.190730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:15.190730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:15.190730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:15.190730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:15.190730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:15.190730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:15.190730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:15.190730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:15.190730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:15.190730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:15.190730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:15.190730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:15.190730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 21:53:15.190730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:15.190730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 359/711] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:15 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:15.197625 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c17fee-460d-47ff-ac99-3daf93d2b2c0 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:15.231531 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:15.233118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:15.246809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428df67-585b-47b4-b796-7a4be8ff46a3 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:15.297398 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f26bb8-a5ce-4ecb-852d-c6ffc4219be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:15.298085 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f26bb8-a5ce-4ecb-852d-c6ffc4219be1 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:15.298374 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f26bb8-a5ce-4ecb-852d-c6ffc4219be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:15.298653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f26bb8-a5ce-4ecb-852d-c6ffc4219be1 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:15.299496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f26bb8-a5ce-4ecb-852d-c6ffc4219be1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:15.299811 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f26bb8-a5ce-4ecb-852d-c6ffc4219be1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:15.301340 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f26bb8-a5ce-4ecb-852d-c6ffc4219be1 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:15.309645 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c17fee-460d-47ff-ac99-3daf93d2b2c0 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:15.324229 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 353/712] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:15.329074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f26bb8-a5ce-4ecb-852d-c6ffc4219be1 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:15.346132 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60836536-7ca5-44c7-b16a-79785f1b3308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:15.346808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60836536-7ca5-44c7-b16a-79785f1b3308 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:15.347092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60836536-7ca5-44c7-b16a-79785f1b3308 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:15.347373 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60836536-7ca5-44c7-b16a-79785f1b3308 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:15.348084 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60836536-7ca5-44c7-b16a-79785f1b3308 None admin] RBAC: Adding query filter params (name=network) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:15.348395 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60836536-7ca5-44c7-b16a-79785f1b3308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:15.349819 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60836536-7ca5-44c7-b16a-79785f1b3308 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:15.355343 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 360/713] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Jul 12 21:53:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:15.360677 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60836536-7ca5-44c7-b16a-79785f1b3308 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:15.465197 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:15.472493 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60836536-7ca5-44c7-b16a-79785f1b3308 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:15.523348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f26bb8-a5ce-4ecb-852d-c6ffc4219be1 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:15.538262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547efdaf-8f22-4816-8855-7ac9b73c69b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:15.539647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547efdaf-8f22-4816-8855-7ac9b73c69b6 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:15.540163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547efdaf-8f22-4816-8855-7ac9b73c69b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:15.540533 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547efdaf-8f22-4816-8855-7ac9b73c69b6 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:15.541338 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547efdaf-8f22-4816-8855-7ac9b73c69b6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:15.541729 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547efdaf-8f22-4816-8855-7ac9b73c69b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:15.543276 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547efdaf-8f22-4816-8855-7ac9b73c69b6 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:15.555890 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 361/714] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Jul 12 21:53:15 2022] GET /identity/v3/services?type=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:15.567890 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fe798-034d-4177-9bfc-c2782eca60c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:15.568761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fe798-034d-4177-9bfc-c2782eca60c0 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:15.569029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fe798-034d-4177-9bfc-c2782eca60c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:15.569310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fe798-034d-4177-9bfc-c2782eca60c0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:15.590138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5fe798-034d-4177-9bfc-c2782eca60c0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:15.591643 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5fe798-034d-4177-9bfc-c2782eca60c0 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:15.616588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-2d5fe798-034d-4177-9bfc-c2782eca60c0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:53:15.616588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:15.616588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:15.616588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:15.616588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:15.616588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:15.616588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:15.616588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:15.616588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:15.616588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:15.616588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:15.616588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:15.616588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:15.616588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:15.616588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:15.616588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:15.618960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:15.621204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:15.621204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:15.621204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:15.621204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:15.621204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:15.621204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:15.621204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:15.621204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:15.621204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:53:15.621204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:15.625145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 354/715] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Jul 12 21:53:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:15.626151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547efdaf-8f22-4816-8855-7ac9b73c69b6 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:15.773630 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547efdaf-8f22-4816-8855-7ac9b73c69b6 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:15.808915 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d625b42-840f-4b90-af4b-574eedcdc984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:15.809591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d625b42-840f-4b90-af4b-574eedcdc984 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:15.809860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d625b42-840f-4b90-af4b-574eedcdc984 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:15.810136 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d625b42-840f-4b90-af4b-574eedcdc984 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:15.810987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d625b42-840f-4b90-af4b-574eedcdc984 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:15.811295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d625b42-840f-4b90-af4b-574eedcdc984 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:15.812742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d625b42-840f-4b90-af4b-574eedcdc984 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:15.831552 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 362/716] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Jul 12 21:53:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:15.838024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fe798-034d-4177-9bfc-c2782eca60c0 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:15.851959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:15.986558 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fe798-034d-4177-9bfc-c2782eca60c0 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:16.043110 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a34f05-9424-4ad4-b243-52dd64602bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:16.043693 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a34f05-9424-4ad4-b243-52dd64602bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:16.044053 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a34f05-9424-4ad4-b243-52dd64602bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:16.044366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a34f05-9424-4ad4-b243-52dd64602bc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:16.137483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a34f05-9424-4ad4-b243-52dd64602bc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:16.138974 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a34f05-9424-4ad4-b243-52dd64602bc4 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:16.156456 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:16.272417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 355/717] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:53:15 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 21:53:16.277996 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d625b42-840f-4b90-af4b-574eedcdc984 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:16.299195 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:16.427689 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d625b42-840f-4b90-af4b-574eedcdc984 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:16.480370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e7d86-70cc-4b8c-aa8f-bf8a9e60abeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:16.481201 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e7d86-70cc-4b8c-aa8f-bf8a9e60abeb None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:16.481487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e7d86-70cc-4b8c-aa8f-bf8a9e60abeb None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:16.481763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e7d86-70cc-4b8c-aa8f-bf8a9e60abeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:16.500553 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e7d86-70cc-4b8c-aa8f-bf8a9e60abeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:16.502051 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e7d86-70cc-4b8c-aa8f-bf8a9e60abeb None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:16.527836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-291e7d86-70cc-4b8c-aa8f-bf8a9e60abeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:16.527836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:16.527836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:16.527836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:16.527836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:16.527836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:16.527836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:16.527836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:16.527836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:16.527836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:16.527836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:16.527836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:16.527836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:16.527836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:16.527836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:16.530139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:16.530139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:16.530139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:16.530139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:16.530139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:16.530139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:16.530139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:16.530139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:16.530139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:16.530139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:16.530139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:16.530139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:16.530139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:16.530139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:16.530139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:16.530139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:16.530139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:16.532300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:16.532300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:16.532300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:16.532300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:16.532300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 363/718] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:16.534856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a34f05-9424-4ad4-b243-52dd64602bc4 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:16.661090 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a34f05-9424-4ad4-b243-52dd64602bc4 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:16.680737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:16.716026 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151df9f-ec8b-494a-90cc-bd4552ce361a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:16.716720 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151df9f-ec8b-494a-90cc-bd4552ce361a None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:16.717026 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151df9f-ec8b-494a-90cc-bd4552ce361a None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:16.717312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151df9f-ec8b-494a-90cc-bd4552ce361a None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:16.718049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151df9f-ec8b-494a-90cc-bd4552ce361a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:16.718360 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151df9f-ec8b-494a-90cc-bd4552ce361a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:16.719749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151df9f-ec8b-494a-90cc-bd4552ce361a None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:16.732336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 356/719] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:16.737706 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e7d86-70cc-4b8c-aa8f-bf8a9e60abeb None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:16.781806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8feeb-9589-42c4-a6a4-4fa73b689e35 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:16.782131 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8feeb-9589-42c4-a6a4-4fa73b689e35 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:16.782407 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8feeb-9589-42c4-a6a4-4fa73b689e35 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:16.783883 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 357/720] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:53:16.792749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c52568-1bb6-4744-b6a2-a97ba3adef2a None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:16.792749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c52568-1bb6-4744-b6a2-a97ba3adef2a None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:16.792749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c52568-1bb6-4744-b6a2-a97ba3adef2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:16.810973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-58c52568-1bb6-4744-b6a2-a97ba3adef2a None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:16.813042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-58c52568-1bb6-4744-b6a2-a97ba3adef2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:16.872768 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:16.906594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 358/721] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:16 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 21:53:16.931878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c52568-1bb6-4744-b6a2-a97ba3adef2a None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:16.957788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e7d86-70cc-4b8c-aa8f-bf8a9e60abeb None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:17.002861 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68f6d6-9957-4004-8851-3fc23489e800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:17.004273 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68f6d6-9957-4004-8851-3fc23489e800 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:17.004767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68f6d6-9957-4004-8851-3fc23489e800 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:17.005249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68f6d6-9957-4004-8851-3fc23489e800 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:17.016316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68f6d6-9957-4004-8851-3fc23489e800 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:17.018780 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68f6d6-9957-4004-8851-3fc23489e800 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:17.039052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-da68f6d6-9957-4004-8851-3fc23489e800 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:17.039052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:17.039052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:17.039052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:17.039052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:17.039052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:17.039052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:17.039052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:17.039052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:17.039052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:17.039052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:17.039052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:17.039052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:17.039052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:17.039052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:17.040427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:17.040427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:17.040427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:17.040427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:17.040427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:17.040427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:17.040427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:17.040427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:17.040427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:17.040427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:17.040427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:17.040427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:17.040427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:17.040427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:17.040427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:17.040427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:17.040427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:17.041368 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:17.041368 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:17.041368 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:17.041368 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:17.041368 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:17.041368 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:17.041368 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:17.041368 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:17.042377 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 364/722] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:17.045188 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0daff-60ec-476d-b2de-87e887246f43 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:17.045188 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0daff-60ec-476d-b2de-87e887246f43 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:17.045538 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0daff-60ec-476d-b2de-87e887246f43 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:17.051230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 365/723] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:17.051230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0daff-60ec-476d-b2de-87e887246f43 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:17.069035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:17.074716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c52568-1bb6-4744-b6a2-a97ba3adef2a None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:17.113898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a4824-79e7-46f7-b381-f7c3b0165398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:17.114788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a4824-79e7-46f7-b381-f7c3b0165398 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:17.115071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a4824-79e7-46f7-b381-f7c3b0165398 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:17.115348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a4824-79e7-46f7-b381-f7c3b0165398 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:17.127765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a4824-79e7-46f7-b381-f7c3b0165398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:17.136639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-9b2a4824-79e7-46f7-b381-f7c3b0165398 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:53:17.136639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:17.136639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:17.136639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:17.136639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:17.136639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:17.136639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:17.136639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:17.136639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:17.136639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:17.136639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:17.136639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:17.136639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:17.136639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:17.136639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:17.136639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:17.137598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:17.137598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:17.137598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:17.137598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:17.137598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:17.137598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:17.137598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:17.137598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:17.137598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:17.137598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:17.137598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:17.137598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:17.137598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:17.137598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:17.137598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:17.137598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:17.138419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:17.138419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:17.138419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:17.138419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:17.138419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:17.138419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:17.138419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:17.138419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:17.138419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:17.138419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:17.138419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:53:17.138419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:17.142902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 359/724] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:53:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:17.144570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8aa08a-3389-4203-a9e5-e7f44d176b57 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:17.144737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8aa08a-3389-4203-a9e5-e7f44d176b57 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:17.144927 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8aa08a-3389-4203-a9e5-e7f44d176b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:17.164644 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-ec8aa08a-3389-4203-a9e5-e7f44d176b57 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:17.165936 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ec8aa08a-3389-4203-a9e5-e7f44d176b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:17.187706 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0daff-60ec-476d-b2de-87e887246f43 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:17.224387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:17.234882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59770a07-678e-46bc-af21-d697aa4dd3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:17.236061 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59770a07-678e-46bc-af21-d697aa4dd3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:17.236750 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59770a07-678e-46bc-af21-d697aa4dd3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:17.237225 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59770a07-678e-46bc-af21-d697aa4dd3f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:17.238773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59770a07-678e-46bc-af21-d697aa4dd3f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:17.239284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59770a07-678e-46bc-af21-d697aa4dd3f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:17.241742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59770a07-678e-46bc-af21-d697aa4dd3f6 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:17.252731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 360/725] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:53:17.252731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8aa08a-3389-4203-a9e5-e7f44d176b57 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:17.268171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 366/726] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:17.274590 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59770a07-678e-46bc-af21-d697aa4dd3f6 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:17.373963 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:17.388020 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8aa08a-3389-4203-a9e5-e7f44d176b57 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:17.396938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59770a07-678e-46bc-af21-d697aa4dd3f6 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:17.437822 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafe51a-26c9-416e-9685-a2baafb622bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:17.439080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafe51a-26c9-416e-9685-a2baafb622bd None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:17.439772 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafe51a-26c9-416e-9685-a2baafb622bd None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:17.440289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafe51a-26c9-416e-9685-a2baafb622bd None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:17.441343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafe51a-26c9-416e-9685-a2baafb622bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:17.442079 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafe51a-26c9-416e-9685-a2baafb622bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:17.443775 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8d50f-e9b5-4607-86f5-c4cdd443ee3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:17.444017 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafe51a-26c9-416e-9685-a2baafb622bd None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:17.444742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8d50f-e9b5-4607-86f5-c4cdd443ee3e None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:17.445647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8d50f-e9b5-4607-86f5-c4cdd443ee3e None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:17.446003 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8d50f-e9b5-4607-86f5-c4cdd443ee3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:17.455741 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 361/727] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:53:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:17.456877 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafe51a-26c9-416e-9685-a2baafb622bd None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:17.464172 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8d50f-e9b5-4607-86f5-c4cdd443ee3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:17.475080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-69f8d50f-e9b5-4607-86f5-c4cdd443ee3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:17.475080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:17.475080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:17.475080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:17.475080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:17.475080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:17.475080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:17.475080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:17.475080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:17.475080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:17.475080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:17.475080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:17.475080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:17.475080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:17.475080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:17.475080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:17.476126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:17.476126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:17.476126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:17.476126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:17.476126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:17.476126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:17.476126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:17.476126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:17.476126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:17.476126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:17.476126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:17.476126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:17.476126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:17.476126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:17.476126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:17.476126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:17.477621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:17.477621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:17.477621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:17.477621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:17.477621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:17.477621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:17.477621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:17.477621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:17.477621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:17.477621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:17.477621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:17.477621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:17.480793 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 367/728] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:53:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:17.482010 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8d50f-e9b5-4607-86f5-c4cdd443ee3e None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:17.542388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:17.581378 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafe51a-26c9-416e-9685-a2baafb622bd None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:17.601988 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8d50f-e9b5-4607-86f5-c4cdd443ee3e None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:17.617548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990077cb-f04c-4b97-ade1-27e8fc58c214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:17.618212 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990077cb-f04c-4b97-ade1-27e8fc58c214 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:17.618495 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990077cb-f04c-4b97-ade1-27e8fc58c214 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:17.618772 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990077cb-f04c-4b97-ade1-27e8fc58c214 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:17.619605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990077cb-f04c-4b97-ade1-27e8fc58c214 None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=24915625354f418eb7bc4d5ada85fdb0) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:17.619916 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990077cb-f04c-4b97-ade1-27e8fc58c214 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:17.621389 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990077cb-f04c-4b97-ade1-27e8fc58c214 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:17.635082 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 362/729] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:53:17 2022] GET /identity/v3/role_assignments?user.id=24915625354f418eb7bc4d5ada85fdb0&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=053d1730ed764973abb1490894e07681 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:17.635082 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990077cb-f04c-4b97-ade1-27e8fc58c214 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:17.651542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72263cf6-f50c-41a5-96b3-8963dc0c1e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:17.652407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72263cf6-f50c-41a5-96b3-8963dc0c1e3a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:17.652703 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72263cf6-f50c-41a5-96b3-8963dc0c1e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:17.653616 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72263cf6-f50c-41a5-96b3-8963dc0c1e3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:17.669095 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:17.671586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72263cf6-f50c-41a5-96b3-8963dc0c1e3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:17.673124 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72263cf6-f50c-41a5-96b3-8963dc0c1e3a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:17.699230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-72263cf6-f50c-41a5-96b3-8963dc0c1e3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:17.699230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:17.699230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:17.699230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:17.699230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:17.699230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:17.699230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:17.699230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:17.699230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:17.699230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:17.699230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:17.699230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:17.699230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:17.699230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:17.699230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:17.700171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:17.700171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:17.700171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:17.700171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:17.700171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:17.700171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:17.700171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:17.700171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:17.700171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:17.700171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:17.700171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:17.700171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:17.700171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:17.700171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:17.700171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:17.700171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:17.700171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:17.701117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:17.701117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:17.701117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:17.701117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:17.704881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 368/730] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:17.708236 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80d171-eae4-4993-b932-50d124fb6614 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:17.708520 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80d171-eae4-4993-b932-50d124fb6614 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:17.708821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80d171-eae4-4993-b932-50d124fb6614 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:17.710561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 369/731] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:17.711818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80d171-eae4-4993-b932-50d124fb6614 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:17.767974 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990077cb-f04c-4b97-ade1-27e8fc58c214 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:17.823103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcc1da-f54f-4864-b574-980333f9632a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:17.824389 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcc1da-f54f-4864-b574-980333f9632a None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:17.825174 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcc1da-f54f-4864-b574-980333f9632a None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:17.825666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcc1da-f54f-4864-b574-980333f9632a None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:17.826972 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcc1da-f54f-4864-b574-980333f9632a None admin] RBAC: Adding query filter params (name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:17.827485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcc1da-f54f-4864-b574-980333f9632a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:17.829794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcc1da-f54f-4864-b574-980333f9632a None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:17.840122 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:17.848762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 363/732] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:53:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:17.848762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5546c0c-013c-4bc2-9759-3f44ddac1ec1 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:17.848762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5546c0c-013c-4bc2-9759-3f44ddac1ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:17.848762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5546c0c-013c-4bc2-9759-3f44ddac1ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:17.852187 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80d171-eae4-4993-b932-50d124fb6614 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:17.874404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-e5546c0c-013c-4bc2-9759-3f44ddac1ec1 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:17.875816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e5546c0c-013c-4bc2-9759-3f44ddac1ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:17.893962 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8fe9f8-64a2-4e67-b804-342a2f548b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:17.895077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fe9f8-64a2-4e67-b804-342a2f548b7d None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:17.895562 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fe9f8-64a2-4e67-b804-342a2f548b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:17.896014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fe9f8-64a2-4e67-b804-342a2f548b7d None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:17.896964 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8fe9f8-64a2-4e67-b804-342a2f548b7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:17.897424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8fe9f8-64a2-4e67-b804-342a2f548b7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:17.899039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8fe9f8-64a2-4e67-b804-342a2f548b7d None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:17.914507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 370/733] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:17.915705 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8fe9f8-64a2-4e67-b804-342a2f548b7d None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:17.977029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 364/734] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:53:17.977029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5546c0c-013c-4bc2-9759-3f44ddac1ec1 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:17.992962 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:18.049630 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8fe9f8-64a2-4e67-b804-342a2f548b7d None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:18.085438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0bbfc-6b7a-4884-96fd-be4185044fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:18.086425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0bbfc-6b7a-4884-96fd-be4185044fdf None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:18.086711 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0bbfc-6b7a-4884-96fd-be4185044fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:18.086992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0bbfc-6b7a-4884-96fd-be4185044fdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:18.093907 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5546c0c-013c-4bc2-9759-3f44ddac1ec1 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:18.098224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0bbfc-6b7a-4884-96fd-be4185044fdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:18.099759 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0bbfc-6b7a-4884-96fd-be4185044fdf None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:18.118461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-21e0bbfc-6b7a-4884-96fd-be4185044fdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:18.118461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:18.118461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:18.118461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:18.118461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:18.118461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:18.118461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:18.118461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:18.118461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:18.118461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:18.118461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:18.118461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:18.118461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:18.118461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:18.118461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:18.119399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:18.119399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:18.119399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:18.119399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:18.119399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:18.119399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:18.119399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:18.119399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:18.119399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:18.119399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:18.119399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:18.119399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:18.119399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:18.119399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:18.119399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:18.119399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:18.119399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:18.120275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:18.120275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:18.120275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:18.120275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:18.120275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 371/735] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:18.121011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40512c-80c7-4ec4-8817-cb7839ad0f6f None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:18.121218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40512c-80c7-4ec4-8817-cb7839ad0f6f None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:18.121523 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40512c-80c7-4ec4-8817-cb7839ad0f6f None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:18.122916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 372/736] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:18.125414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40512c-80c7-4ec4-8817-cb7839ad0f6f None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:18.141079 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f0323-ad76-4785-900e-ad1d298ae621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:18.141935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f0323-ad76-4785-900e-ad1d298ae621 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:18.142284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f0323-ad76-4785-900e-ad1d298ae621 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:18.142726 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f0323-ad76-4785-900e-ad1d298ae621 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:18.142871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:18.152292 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f0323-ad76-4785-900e-ad1d298ae621 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:18.154108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f0323-ad76-4785-900e-ad1d298ae621 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:18.170005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-973f0323-ad76-4785-900e-ad1d298ae621 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 21:53:18.170005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:18.170005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:18.170005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:18.170005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:18.170005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:18.170005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:18.170005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:18.170005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:18.170005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:18.170005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:18.170005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:18.170005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:18.170005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:18.170005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:18.170005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:18.171002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:18.172016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:18.172016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:18.172016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:18.172016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:18.172016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:18.172016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:18.172016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:18.172016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:18.172016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 21:53:18.172016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:18.174007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 365/737] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:53:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:18.174936 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f0323-ad76-4785-900e-ad1d298ae621 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:18.246926 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:18.254665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40512c-80c7-4ec4-8817-cb7839ad0f6f None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:18.283000 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f0323-ad76-4785-900e-ad1d298ae621 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:18.304622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397d031-96ff-4cdd-a11e-b4dea9c4d073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:18.305623 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397d031-96ff-4cdd-a11e-b4dea9c4d073 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:18.305908 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397d031-96ff-4cdd-a11e-b4dea9c4d073 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:18.306187 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397d031-96ff-4cdd-a11e-b4dea9c4d073 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:18.306837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397d031-96ff-4cdd-a11e-b4dea9c4d073 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:18.308240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397d031-96ff-4cdd-a11e-b4dea9c4d073 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:18.320846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-f397d031-96ff-4cdd-a11e-b4dea9c4d073 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 21:53:18.320846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:18.320846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:18.320846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:18.320846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:18.320846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:18.320846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:18.320846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:18.320846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 21:53:18.320846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:18.320846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:18.320846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:18.320846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:18.320846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:18.320846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:18.320846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:18.321981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:18.321981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:18.321981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:18.321981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:18.321981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:18.321981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:18.321981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:18.321981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:18.321981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:18.321981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:18.321981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:18.321981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:18.321981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:18.321981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:18.321981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:18.321981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:18.322818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:18.322818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:18.322818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:18.322818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:18.322818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:18.322818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:18.322818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:18.322818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:18.322818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:18.322818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:18.322818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:18.322818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:18.322818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:18.322818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 21:53:18.322818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:18.322818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 373/738] 10.222.0.44 () {56 vars in 1129 bytes} [Tue Jul 12 21:53:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:18.323675 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc25bc-e683-402a-b189-d38472b48ba5 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:18.323745 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc25bc-e683-402a-b189-d38472b48ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:18.324423 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc25bc-e683-402a-b189-d38472b48ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:18.329093 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df88fe-0a6a-4774-a4d6-8bc70db1f7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:18.329769 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df88fe-0a6a-4774-a4d6-8bc70db1f7fb None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:18.330059 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df88fe-0a6a-4774-a4d6-8bc70db1f7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:18.330338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df88fe-0a6a-4774-a4d6-8bc70db1f7fb None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:18.331071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df88fe-0a6a-4774-a4d6-8bc70db1f7fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:18.331382 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df88fe-0a6a-4774-a4d6-8bc70db1f7fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:18.332812 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df88fe-0a6a-4774-a4d6-8bc70db1f7fb None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:18.343133 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-b6fc25bc-e683-402a-b189-d38472b48ba5 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:18.344564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b6fc25bc-e683-402a-b189-d38472b48ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:18.345692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 366/739] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:18.346523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df88fe-0a6a-4774-a4d6-8bc70db1f7fb None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:18.423020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 374/740] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 21:53:18.424833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc25bc-e683-402a-b189-d38472b48ba5 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:18.447268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:18.479031 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df88fe-0a6a-4774-a4d6-8bc70db1f7fb None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:18.506429 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:18.515469 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1b5a9-c18d-4575-83ba-16e2a0b94e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:18.516143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1b5a9-c18d-4575-83ba-16e2a0b94e30 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:18.516428 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1b5a9-c18d-4575-83ba-16e2a0b94e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:18.516704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1b5a9-c18d-4575-83ba-16e2a0b94e30 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:18.517565 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1b5a9-c18d-4575-83ba-16e2a0b94e30 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:18.517873 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1b5a9-c18d-4575-83ba-16e2a0b94e30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:18.519296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1b5a9-c18d-4575-83ba-16e2a0b94e30 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:18.538852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 367/741] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Jul 12 21:53:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:18.539857 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1b5a9-c18d-4575-83ba-16e2a0b94e30 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:18.556576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc25bc-e683-402a-b189-d38472b48ba5 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:18.593247 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487e063-79e5-44e9-95a6-858341ca2aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:18.593918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487e063-79e5-44e9-95a6-858341ca2aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:18.594208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487e063-79e5-44e9-95a6-858341ca2aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:18.594484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487e063-79e5-44e9-95a6-858341ca2aa3 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:18.595189 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487e063-79e5-44e9-95a6-858341ca2aa3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:18.595497 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487e063-79e5-44e9-95a6-858341ca2aa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:18.596893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487e063-79e5-44e9-95a6-858341ca2aa3 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:18.606387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 375/742] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Jul 12 21:53:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:18.606387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487e063-79e5-44e9-95a6-858341ca2aa3 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:18.652512 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1b5a9-c18d-4575-83ba-16e2a0b94e30 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:18.683935 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:18.690342 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34b5f5-7b22-46e7-aa90-a2b9e74f3df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:18.691735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34b5f5-7b22-46e7-aa90-a2b9e74f3df9 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:18.692019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34b5f5-7b22-46e7-aa90-a2b9e74f3df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:18.692293 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34b5f5-7b22-46e7-aa90-a2b9e74f3df9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:18.700170 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34b5f5-7b22-46e7-aa90-a2b9e74f3df9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:18.701655 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34b5f5-7b22-46e7-aa90-a2b9e74f3df9 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:18.716662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-3a34b5f5-7b22-46e7-aa90-a2b9e74f3df9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 21:53:18.716662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:18.716662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:18.716662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:18.716662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:18.716662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:18.716662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:18.716662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:18.716662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:18.716662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:18.716662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:18.716662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:18.716662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:18.716662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:18.716662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:18.716662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:18.722562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:18.723781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:18.723781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:18.723781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:18.723781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:18.723781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:18.723781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:18.723781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:18.723781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:18.723781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 21:53:18.723781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:18.723781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 368/743] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Jul 12 21:53:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:18.723781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34b5f5-7b22-46e7-aa90-a2b9e74f3df9 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:18.727527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487e063-79e5-44e9-95a6-858341ca2aa3 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:18.776781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c01e67-4395-4efe-9e61-a71202e97b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:18.777457 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c01e67-4395-4efe-9e61-a71202e97b2d None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:18.777738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c01e67-4395-4efe-9e61-a71202e97b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:18.778016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c01e67-4395-4efe-9e61-a71202e97b2d None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:18.778751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c01e67-4395-4efe-9e61-a71202e97b2d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:18.780158 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c01e67-4395-4efe-9e61-a71202e97b2d None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:18.783337 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:18.816742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 376/744] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Jul 12 21:53:18 2022] POST /identity/v3/services => generated 240 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:53:18.816742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c01e67-4395-4efe-9e61-a71202e97b2d None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:18.848645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34b5f5-7b22-46e7-aa90-a2b9e74f3df9 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:18.898663 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bef514-b741-4c92-953f-f3d50d1fd1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:18.899380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bef514-b741-4c92-953f-f3d50d1fd1cd None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:18.899664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bef514-b741-4c92-953f-f3d50d1fd1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:18.899938 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bef514-b741-4c92-953f-f3d50d1fd1cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:18.975026 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c01e67-4395-4efe-9e61-a71202e97b2d None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:18.995838 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bef514-b741-4c92-953f-f3d50d1fd1cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:18.997384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bef514-b741-4c92-953f-f3d50d1fd1cd None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:19.011363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:19.026028 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de645cde-d988-4bb5-bcbd-d3d84e44c6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:19.027209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de645cde-d988-4bb5-bcbd-d3d84e44c6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:19.027993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de645cde-d988-4bb5-bcbd-d3d84e44c6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:19.028703 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de645cde-d988-4bb5-bcbd-d3d84e44c6d7 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:19.029829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de645cde-d988-4bb5-bcbd-d3d84e44c6d7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:19.030604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de645cde-d988-4bb5-bcbd-d3d84e44c6d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:19.032612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de645cde-d988-4bb5-bcbd-d3d84e44c6d7 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:19.039159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 377/745] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Jul 12 21:53:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:19.043725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de645cde-d988-4bb5-bcbd-d3d84e44c6d7 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:19.049107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:19.121228 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 369/746] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:53:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:53:19.122492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bef514-b741-4c92-953f-f3d50d1fd1cd None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:19.192328 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de645cde-d988-4bb5-bcbd-d3d84e44c6d7 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:19.230005 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decd146-fba2-46f0-946d-29b459537e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:19.230776 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decd146-fba2-46f0-946d-29b459537e79 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:19.231126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decd146-fba2-46f0-946d-29b459537e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:19.231470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decd146-fba2-46f0-946d-29b459537e79 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:19.232382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3decd146-fba2-46f0-946d-29b459537e79 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:19.232808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3decd146-fba2-46f0-946d-29b459537e79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:19.234456 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3decd146-fba2-46f0-946d-29b459537e79 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:19.249062 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bef514-b741-4c92-953f-f3d50d1fd1cd None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:19.255244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 378/747] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Jul 12 21:53:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:19.256266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decd146-fba2-46f0-946d-29b459537e79 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:19.288760 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5048d2-eef5-42e9-bb39-f0dac2e62ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:19.289720 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5048d2-eef5-42e9-bb39-f0dac2e62ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:19.290289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5048d2-eef5-42e9-bb39-f0dac2e62ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:19.290744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5048d2-eef5-42e9-bb39-f0dac2e62ee8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:19.291803 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5048d2-eef5-42e9-bb39-f0dac2e62ee8 None admin] RBAC: Adding query filter params (interface=public, service_id=6e52b092b50541278075b80fa25f875a, region_id=RegionOne) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:19.292324 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5048d2-eef5-42e9-bb39-f0dac2e62ee8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:19.293934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5048d2-eef5-42e9-bb39-f0dac2e62ee8 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:19.307134 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 370/748] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Jul 12 21:53:19 2022] GET /identity/v3/endpoints?service_id=6e52b092b50541278075b80fa25f875a&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:19.308437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5048d2-eef5-42e9-bb39-f0dac2e62ee8 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:19.314562 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:19.322192 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:19.367729 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decd146-fba2-46f0-946d-29b459537e79 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:19.405471 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37f2e1-eabc-403c-934e-318b38615920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:19.406512 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37f2e1-eabc-403c-934e-318b38615920 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:19.406941 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37f2e1-eabc-403c-934e-318b38615920 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:19.407383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37f2e1-eabc-403c-934e-318b38615920 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:19.419319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37f2e1-eabc-403c-934e-318b38615920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:19.421014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37f2e1-eabc-403c-934e-318b38615920 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:19.425607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5048d2-eef5-42e9-bb39-f0dac2e62ee8 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:19.440390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-2c37f2e1-eabc-403c-934e-318b38615920 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:19.440390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:19.440390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:19.440390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:19.440390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:19.440390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:19.440390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:19.440390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:19.440390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:19.440390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:19.440390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:19.440390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:19.440390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:19.440390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:19.440390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:19.442015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:19.442015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:19.442015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:19.442015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:19.442015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:19.442015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:19.442015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:19.442015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:19.442015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:19.442015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:19.442015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:19.442015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:19.442015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:19.442015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:19.442015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:19.442015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:19.442015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:19.443053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:19.443053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:19.443053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:19.443053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:19.445832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 379/749] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:19.446958 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37f2e1-eabc-403c-934e-318b38615920 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:19.474705 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0c9e7-06ce-4c95-bd06-41686c8b8fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:19.475618 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0c9e7-06ce-4c95-bd06-41686c8b8fbf None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:19.476184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0c9e7-06ce-4c95-bd06-41686c8b8fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:19.476566 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0c9e7-06ce-4c95-bd06-41686c8b8fbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:19.556375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0c9e7-06ce-4c95-bd06-41686c8b8fbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:19.557898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37f2e1-eabc-403c-934e-318b38615920 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:19.558040 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0c9e7-06ce-4c95-bd06-41686c8b8fbf None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:19.560696 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:19.569183 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:19.598992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f23f38-fd2f-4e9a-8860-5e4b8cfb0139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:19.598992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f23f38-fd2f-4e9a-8860-5e4b8cfb0139 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:19.599420 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f23f38-fd2f-4e9a-8860-5e4b8cfb0139 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:19.599609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f23f38-fd2f-4e9a-8860-5e4b8cfb0139 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:19.600252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f23f38-fd2f-4e9a-8860-5e4b8cfb0139 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:19.602076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f23f38-fd2f-4e9a-8860-5e4b8cfb0139 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:19.612779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 380/750] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Jul 12 21:53:19 2022] GET /identity/v3/services => generated 772 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:19.612779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f23f38-fd2f-4e9a-8860-5e4b8cfb0139 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:19.720693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f23f38-fd2f-4e9a-8860-5e4b8cfb0139 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:19.727837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 371/751] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:53:19 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:53:19.733331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0c9e7-06ce-4c95-bd06-41686c8b8fbf None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:19.759741 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23290f35-3c5f-42e3-b6e1-08f342582cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:19.760532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23290f35-3c5f-42e3-b6e1-08f342582cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:19.760855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23290f35-3c5f-42e3-b6e1-08f342582cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:19.761136 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23290f35-3c5f-42e3-b6e1-08f342582cf0 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:19.761939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23290f35-3c5f-42e3-b6e1-08f342582cf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:19.762255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23290f35-3c5f-42e3-b6e1-08f342582cf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:19.763803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23290f35-3c5f-42e3-b6e1-08f342582cf0 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:19.778359 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 381/752] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:19.779995 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be6ac2-d3b4-4177-89f8-6f397d865dc6 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:19.780317 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be6ac2-d3b4-4177-89f8-6f397d865dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:19.780687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be6ac2-d3b4-4177-89f8-6f397d865dc6 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:19.782581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 382/753] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:19.783379 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be6ac2-d3b4-4177-89f8-6f397d865dc6 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:19.821288 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:19.848937 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0c9e7-06ce-4c95-bd06-41686c8b8fbf None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:19.858556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:19.890774 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb403d-8601-47c6-810d-a66c11c0a291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:19.891695 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb403d-8601-47c6-810d-a66c11c0a291 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:19.892050 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb403d-8601-47c6-810d-a66c11c0a291 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:19.892397 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb403d-8601-47c6-810d-a66c11c0a291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:19.904064 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb403d-8601-47c6-810d-a66c11c0a291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:19.905585 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb403d-8601-47c6-810d-a66c11c0a291 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:19.906627 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be6ac2-d3b4-4177-89f8-6f397d865dc6 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:19.925069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-57fb403d-8601-47c6-810d-a66c11c0a291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:19.925069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:19.925069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:19.925069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:19.925069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:19.925069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:19.925069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:19.925069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:19.925069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:19.925069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:19.925069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:19.925069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:19.925069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:19.925069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:19.925069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:19.925998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:19.925998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:19.925998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:19.925998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:19.925998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:19.925998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:19.925998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:19.925998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:19.925998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:19.925998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:19.925998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:19.925998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:19.925998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:19.925998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:19.925998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:19.925998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:19.925998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:19.926873 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:19.926873 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:19.926873 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:19.926873 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:19.926873 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 372/754] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:19.927834 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dab44-1131-4f7c-8bc9-7ddafb53c286 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:19.928128 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dab44-1131-4f7c-8bc9-7ddafb53c286 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:19.928638 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dab44-1131-4f7c-8bc9-7ddafb53c286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:19.948604 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-233dab44-1131-4f7c-8bc9-7ddafb53c286 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:19.950273 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-233dab44-1131-4f7c-8bc9-7ddafb53c286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:19.950457 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69141e17-87f7-4b69-bd69-f11ae1665d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:19.951256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69141e17-87f7-4b69-bd69-f11ae1665d4e None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:19.951542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69141e17-87f7-4b69-bd69-f11ae1665d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:19.951819 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69141e17-87f7-4b69-bd69-f11ae1665d4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:19.967244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69141e17-87f7-4b69-bd69-f11ae1665d4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:19.968836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69141e17-87f7-4b69-bd69-f11ae1665d4e None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:19.985064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-69141e17-87f7-4b69-bd69-f11ae1665d4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:19.985064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:19.985064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:19.985064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:19.985064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:19.985064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:19.985064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:19.985064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:19.985064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:19.985064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:19.985064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:19.985064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:19.985064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:19.985064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:19.985064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:19.985985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:19.985985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:19.985985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:19.985985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:19.985985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:19.985985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:19.985985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:19.985985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:19.985985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:19.985985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:19.985985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:19.985985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:19.985985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:19.985985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:19.985985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:19.985985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:19.985985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:19.986878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:19.986878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:19.986878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:19.986878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:19.986878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:19.986878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:19.986878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:19.986878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:19.986878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 383/755] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:19.987816 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69141e17-87f7-4b69-bd69-f11ae1665d4e None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:20.036848 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 373/756] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:20.038073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233dab44-1131-4f7c-8bc9-7ddafb53c286 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:20.086710 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:20.101890 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69141e17-87f7-4b69-bd69-f11ae1665d4e None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:20.141989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a66ebe-d431-44aa-a1fa-65c1575f424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:20.142666 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a66ebe-d431-44aa-a1fa-65c1575f424c None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:20.142949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a66ebe-d431-44aa-a1fa-65c1575f424c None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:20.143223 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a66ebe-d431-44aa-a1fa-65c1575f424c None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:20.143954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a66ebe-d431-44aa-a1fa-65c1575f424c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:20.144259 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a66ebe-d431-44aa-a1fa-65c1575f424c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:20.145718 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a66ebe-d431-44aa-a1fa-65c1575f424c None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:20.170848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 384/757] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:20.170848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a66ebe-d431-44aa-a1fa-65c1575f424c None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:20.293481 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a66ebe-d431-44aa-a1fa-65c1575f424c None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:20.313921 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:20.330423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233dab44-1131-4f7c-8bc9-7ddafb53c286 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:20.332679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afdc5d-6dfa-431a-8bac-83cc74c3511d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:20.333783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afdc5d-6dfa-431a-8bac-83cc74c3511d None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:20.334298 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afdc5d-6dfa-431a-8bac-83cc74c3511d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:20.334780 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afdc5d-6dfa-431a-8bac-83cc74c3511d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:20.346926 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afdc5d-6dfa-431a-8bac-83cc74c3511d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:20.355470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-b3afdc5d-6dfa-431a-8bac-83cc74c3511d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:53:20.355470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:20.355470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:20.355470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:20.355470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:20.355470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:20.355470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:20.355470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:20.355470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:20.355470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:20.355470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:20.355470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:20.355470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:20.355470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:20.355470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:20.355470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:20.356394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:20.356394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:20.356394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:20.356394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:20.356394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:20.356394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:20.356394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:20.356394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:20.356394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:20.356394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:20.356394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:20.356394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:20.356394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:20.356394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:20.356394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:20.356394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:20.357560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:20.357560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:20.357560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:20.357560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:20.357560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:20.357560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:20.357560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:20.357560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:20.357560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:20.357560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:20.357560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:53:20.357560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:20.365282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 385/758] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:53:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:20.366363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afdc5d-6dfa-431a-8bac-83cc74c3511d None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:20.367641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d24349-6ad1-4b71-9219-4b993ed48fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:20.368314 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d24349-6ad1-4b71-9219-4b993ed48fae None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:20.368592 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d24349-6ad1-4b71-9219-4b993ed48fae None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:20.368907 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d24349-6ad1-4b71-9219-4b993ed48fae None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:20.369701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d24349-6ad1-4b71-9219-4b993ed48fae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:20.370004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d24349-6ad1-4b71-9219-4b993ed48fae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:20.371450 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d24349-6ad1-4b71-9219-4b993ed48fae None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:20.380923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:20.386785 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 374/759] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:20.387867 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d24349-6ad1-4b71-9219-4b993ed48fae None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:20.499074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d24349-6ad1-4b71-9219-4b993ed48fae None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:20.511424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afdc5d-6dfa-431a-8bac-83cc74c3511d None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:20.542806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8e446-3cd4-4981-bda4-941963f01f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:20.543838 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8e446-3cd4-4981-bda4-941963f01f87 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:20.544351 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8e446-3cd4-4981-bda4-941963f01f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:20.544925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8e446-3cd4-4981-bda4-941963f01f87 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:20.545986 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8e446-3cd4-4981-bda4-941963f01f87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:20.546497 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8e446-3cd4-4981-bda4-941963f01f87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:20.548271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8e446-3cd4-4981-bda4-941963f01f87 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:20.553674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ba1a9-abef-4842-b690-0434b5d58af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:20.555522 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ba1a9-abef-4842-b690-0434b5d58af5 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:20.555798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ba1a9-abef-4842-b690-0434b5d58af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:20.556074 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ba1a9-abef-4842-b690-0434b5d58af5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:20.564760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ba1a9-abef-4842-b690-0434b5d58af5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:20.565341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 375/760] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:53:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:20.565523 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ba1a9-abef-4842-b690-0434b5d58af5 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:20.566882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8e446-3cd4-4981-bda4-941963f01f87 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:20.581607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:20.584184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-bc8ba1a9-abef-4842-b690-0434b5d58af5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:20.584184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:20.584184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:20.584184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:20.584184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:20.584184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:20.584184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:20.584184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:20.584184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:20.584184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:20.584184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:20.584184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:20.584184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:20.584184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:20.584184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:20.586109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:20.586109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:20.586109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:20.586109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:20.586109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:20.586109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:20.586109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:20.586109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:20.586109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:20.586109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:20.586109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:20.586109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:20.586109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:20.586109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:20.586109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:20.586109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:20.586109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:20.586993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:20.586993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:20.586993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:20.586993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:20.586993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:20.586993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:20.586993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:20.586993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:20.590841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 386/761] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:20.592148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ba1a9-abef-4842-b690-0434b5d58af5 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:20.687530 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:20.689871 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8e446-3cd4-4981-bda4-941963f01f87 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:20.722606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ba1a9-abef-4842-b690-0434b5d58af5 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:20.730297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12886438-6296-4e6e-a41d-5c1a10f35e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:20.731756 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12886438-6296-4e6e-a41d-5c1a10f35e9a None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:20.732270 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12886438-6296-4e6e-a41d-5c1a10f35e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:20.732798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12886438-6296-4e6e-a41d-5c1a10f35e9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:20.733959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12886438-6296-4e6e-a41d-5c1a10f35e9a None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=837bb5ac18554dfdbbcc776689639494) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:20.734513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12886438-6296-4e6e-a41d-5c1a10f35e9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:20.736283 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12886438-6296-4e6e-a41d-5c1a10f35e9a None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:20.743585 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 376/762] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:53:20 2022] GET /identity/v3/role_assignments?user.id=837bb5ac18554dfdbbcc776689639494&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=053d1730ed764973abb1490894e07681 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:20.744785 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12886438-6296-4e6e-a41d-5c1a10f35e9a None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:20.762490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9fc2d-7184-4096-ab2b-11b1c72d6a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:20.763643 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9fc2d-7184-4096-ab2b-11b1c72d6a64 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:20.763928 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9fc2d-7184-4096-ab2b-11b1c72d6a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:20.764207 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9fc2d-7184-4096-ab2b-11b1c72d6a64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:20.778974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d9fc2d-7184-4096-ab2b-11b1c72d6a64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:20.781304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d9fc2d-7184-4096-ab2b-11b1c72d6a64 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:20.807139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-c8d9fc2d-7184-4096-ab2b-11b1c72d6a64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:20.807139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:20.807139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:20.807139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:20.807139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:20.807139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:20.807139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:20.807139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:20.807139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:20.807139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:20.807139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:20.807139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:20.807139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:20.807139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:20.807139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:20.808095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:20.808095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:20.808095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:20.808095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:20.808095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:20.808095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:20.808095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:20.808095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:20.808095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:20.808095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:20.808095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:20.808095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:20.808095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:20.808095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:20.808095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:20.808095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:20.808095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:20.809025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:20.809025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:20.809025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:20.809025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:20.809025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 387/763] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:20.821810 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9fc2d-7184-4096-ab2b-11b1c72d6a64 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:20.872839 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:20.930879 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12886438-6296-4e6e-a41d-5c1a10f35e9a None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:20.995801 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d8cc1-7984-420f-86af-1f483f36fe30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:20.997085 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d8cc1-7984-420f-86af-1f483f36fe30 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:20.997439 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d8cc1-7984-420f-86af-1f483f36fe30 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:20.997783 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d8cc1-7984-420f-86af-1f483f36fe30 None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:20.998695 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d8cc1-7984-420f-86af-1f483f36fe30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:20.999069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d8cc1-7984-420f-86af-1f483f36fe30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:21.001042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d8cc1-7984-420f-86af-1f483f36fe30 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:21.017265 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 377/764] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:21.025764 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d8cc1-7984-420f-86af-1f483f36fe30 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:21.033647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9fc2d-7184-4096-ab2b-11b1c72d6a64 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:21.078113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:21.108673 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea48e9-26d6-419f-a6b2-3a83092e4889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:21.109764 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea48e9-26d6-419f-a6b2-3a83092e4889 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:21.110306 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea48e9-26d6-419f-a6b2-3a83092e4889 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:21.110818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea48e9-26d6-419f-a6b2-3a83092e4889 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:21.111773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea48e9-26d6-419f-a6b2-3a83092e4889 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:21.112313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea48e9-26d6-419f-a6b2-3a83092e4889 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:21.114052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea48e9-26d6-419f-a6b2-3a83092e4889 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:21.128704 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 388/765] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:21.134330 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73e1c8-40d4-457b-a6b9-5e59ba478f6c None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:21.134488 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73e1c8-40d4-457b-a6b9-5e59ba478f6c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:21.134925 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73e1c8-40d4-457b-a6b9-5e59ba478f6c None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:21.136118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 389/766] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:21.137524 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73e1c8-40d4-457b-a6b9-5e59ba478f6c None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:21.184031 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d8cc1-7984-420f-86af-1f483f36fe30 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:21.221045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e6a89-9dcb-4ecc-b32e-fe4100c97eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:21.222064 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e6a89-9dcb-4ecc-b32e-fe4100c97eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:21.222605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e6a89-9dcb-4ecc-b32e-fe4100c97eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:21.223081 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e6a89-9dcb-4ecc-b32e-fe4100c97eb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:21.224176 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e6a89-9dcb-4ecc-b32e-fe4100c97eb4 None admin] RBAC: Adding query filter params (role.id=f1e42ea5a1c440bba915b459d2744858, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=8d6b823cc7dc434986f77739fa0259ea) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:21.224693 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e6a89-9dcb-4ecc-b32e-fe4100c97eb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:21.226673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e6a89-9dcb-4ecc-b32e-fe4100c97eb4 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:21.228889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:21.233982 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 378/767] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:53:21 2022] GET /identity/v3/role_assignments?user.id=8d6b823cc7dc434986f77739fa0259ea&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=f1e42ea5a1c440bba915b459d2744858 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:21.243380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b6b80-546e-4529-a085-fa3bf024c641 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:21.243998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b6b80-546e-4529-a085-fa3bf024c641 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:21.244511 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b6b80-546e-4529-a085-fa3bf024c641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:21.263990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73e1c8-40d4-457b-a6b9-5e59ba478f6c None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:21.277820 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-883b6b80-546e-4529-a085-fa3bf024c641 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:21.279429 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-883b6b80-546e-4529-a085-fa3bf024c641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:21.307507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbd09f-ac8a-40fc-a2cc-01a0224b0950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:21.308390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbd09f-ac8a-40fc-a2cc-01a0224b0950 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:21.308676 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbd09f-ac8a-40fc-a2cc-01a0224b0950 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:21.308974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbd09f-ac8a-40fc-a2cc-01a0224b0950 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:21.318662 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbd09f-ac8a-40fc-a2cc-01a0224b0950 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:21.320259 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbd09f-ac8a-40fc-a2cc-01a0224b0950 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:21.336614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-90cbd09f-ac8a-40fc-a2cc-01a0224b0950 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:53:21.336614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:21.336614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:21.336614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:21.336614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:21.336614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:21.336614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:21.336614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:21.336614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:21.336614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:21.336614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:21.336614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:21.336614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:21.336614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:21.336614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:21.336614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:21.337674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:21.338609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:21.338609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:21.338609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:21.338609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:21.338609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:21.338609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:21.338609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:21.338609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:21.338609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:53:21.338609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:21.349256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 390/768] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Jul 12 21:53:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:21.349256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbd09f-ac8a-40fc-a2cc-01a0224b0950 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:21.391856 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:21.424736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 379/769] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:21.484681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbd09f-ac8a-40fc-a2cc-01a0224b0950 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:21.485962 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b6b80-546e-4529-a085-fa3bf024c641 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:21.526499 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a20196-a026-4d89-a24b-451c6d7ae810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:21.527775 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a20196-a026-4d89-a24b-451c6d7ae810 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:21.528308 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a20196-a026-4d89-a24b-451c6d7ae810 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:21.528743 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a20196-a026-4d89-a24b-451c6d7ae810 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:21.529832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a20196-a026-4d89-a24b-451c6d7ae810 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:21.530283 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a20196-a026-4d89-a24b-451c6d7ae810 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:21.531891 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a20196-a026-4d89-a24b-451c6d7ae810 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:21.556508 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 391/770] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Jul 12 21:53:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:21.558144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a20196-a026-4d89-a24b-451c6d7ae810 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:21.608395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b6b80-546e-4529-a085-fa3bf024c641 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:21.637666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:21.658840 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d37022-e451-4fbc-8425-10e8c64bf973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:21.659656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d37022-e451-4fbc-8425-10e8c64bf973 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:21.659942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d37022-e451-4fbc-8425-10e8c64bf973 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:21.660223 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d37022-e451-4fbc-8425-10e8c64bf973 None admin] PATH_INFO: `/v3/services/network` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:21.660922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d37022-e451-4fbc-8425-10e8c64bf973 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:21.662497 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d37022-e451-4fbc-8425-10e8c64bf973 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:21.668294 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:21.672609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a20196-a026-4d89-a24b-451c6d7ae810 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:21.692231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-38d37022-e451-4fbc-8425-10e8c64bf973 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 21:53:21.692231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:21.692231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:21.692231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:21.692231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:21.692231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:21.692231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:21.692231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:21.692231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 21:53:21.692231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:21.692231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:21.692231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:21.692231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:21.692231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:21.692231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:21.692231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:21.692231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:21.693331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:21.693331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:21.693331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:21.693331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:21.693331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:21.693331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:21.693331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:21.693331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:21.693331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:21.693331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:21.693331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:21.693331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:21.693331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:21.693331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:21.693331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:21.693331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:21.695182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:21.695182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:21.695182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:21.695182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:21.695182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:21.695182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:21.695182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:21.695182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:21.695182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:21.695182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:21.695182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:21.695182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:21.695182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 21:53:21.695182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:21.695182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 380/771] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:21 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:21.703198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d37022-e451-4fbc-8425-10e8c64bf973 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:21.713165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5f456-3516-495d-bad9-5e0e008dafba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:21.714268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5f456-3516-495d-bad9-5e0e008dafba None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:21.714778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5f456-3516-495d-bad9-5e0e008dafba None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:21.715265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5f456-3516-495d-bad9-5e0e008dafba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:21.728150 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5f456-3516-495d-bad9-5e0e008dafba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:21.730027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5f456-3516-495d-bad9-5e0e008dafba None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:21.749837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-e7b5f456-3516-495d-bad9-5e0e008dafba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:21.749837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:21.749837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:21.749837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:21.749837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:21.749837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:21.749837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:21.749837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:21.749837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:21.749837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:21.749837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:21.749837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:21.749837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:21.749837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:21.749837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:21.750749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:21.750749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:21.750749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:21.750749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:21.750749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:21.750749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:21.750749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:21.750749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:21.750749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:21.750749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:21.750749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:21.750749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:21.750749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:21.750749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:21.750749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:21.750749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:21.750749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:21.751612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:21.751612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:21.751612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:21.751612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:21.759189 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 392/772] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:21.760774 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5f456-3516-495d-bad9-5e0e008dafba None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:21.870019 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5f456-3516-495d-bad9-5e0e008dafba None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:21.911719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfb4dd-b3bc-4e73-9a03-49f1c05f1d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:21.913405 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfb4dd-b3bc-4e73-9a03-49f1c05f1d65 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:21.913893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfb4dd-b3bc-4e73-9a03-49f1c05f1d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:21.914587 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfb4dd-b3bc-4e73-9a03-49f1c05f1d65 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:21.915496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddfb4dd-b3bc-4e73-9a03-49f1c05f1d65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:21.916168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddfb4dd-b3bc-4e73-9a03-49f1c05f1d65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:21.918104 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddfb4dd-b3bc-4e73-9a03-49f1c05f1d65 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:21.940759 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 393/773] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:21.943172 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc062c-9728-4481-92d0-a2161ca26332 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:21.943922 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc062c-9728-4481-92d0-a2161ca26332 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:21.944942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc062c-9728-4481-92d0-a2161ca26332 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:21.948002 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 394/774] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:21.949371 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc062c-9728-4481-92d0-a2161ca26332 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:21.954171 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d37022-e451-4fbc-8425-10e8c64bf973 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:21.973906 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:22.003974 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396a321-ca50-4383-b9e0-ea1d4a1245c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:22.005023 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396a321-ca50-4383-b9e0-ea1d4a1245c9 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:22.005522 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396a321-ca50-4383-b9e0-ea1d4a1245c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:22.005994 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396a321-ca50-4383-b9e0-ea1d4a1245c9 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:22.006901 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f396a321-ca50-4383-b9e0-ea1d4a1245c9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:22.007435 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f396a321-ca50-4383-b9e0-ea1d4a1245c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:22.009088 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f396a321-ca50-4383-b9e0-ea1d4a1245c9 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:22.014861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 381/775] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Jul 12 21:53:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:22.016519 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28aea57-6b50-4261-b140-4b4e1f2363e1 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:22.017037 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28aea57-6b50-4261-b140-4b4e1f2363e1 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:22.017528 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28aea57-6b50-4261-b140-4b4e1f2363e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:22.044423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-d28aea57-6b50-4261-b140-4b4e1f2363e1 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:22.045709 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d28aea57-6b50-4261-b140-4b4e1f2363e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:22.088298 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc062c-9728-4481-92d0-a2161ca26332 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:22.102851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:22.129033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 382/776] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:22.130153 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963e5f4-05d0-4ad7-ac4b-09fb446a6a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:22.130813 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28aea57-6b50-4261-b140-4b4e1f2363e1 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:22.130942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963e5f4-05d0-4ad7-ac4b-09fb446a6a20 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:22.131205 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963e5f4-05d0-4ad7-ac4b-09fb446a6a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:22.131483 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963e5f4-05d0-4ad7-ac4b-09fb446a6a20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:22.138777 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963e5f4-05d0-4ad7-ac4b-09fb446a6a20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:22.141252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963e5f4-05d0-4ad7-ac4b-09fb446a6a20 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:22.158240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-6963e5f4-05d0-4ad7-ac4b-09fb446a6a20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:22.158240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:22.158240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:22.158240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:22.158240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:22.158240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:22.158240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:22.158240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:22.158240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:22.158240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:22.158240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:22.158240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:22.158240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:22.158240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:22.158240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:22.159360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:22.159360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:22.159360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:22.159360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:22.159360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:22.159360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:22.159360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:22.159360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:22.159360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:22.159360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:22.159360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:22.159360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:22.159360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:22.159360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:22.159360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:22.159360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:22.159360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:22.160322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:22.160322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:22.160322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:22.160322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:22.160322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:22.160322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:22.160322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:22.160322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:22.160322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 395/777] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:22.168825 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963e5f4-05d0-4ad7-ac4b-09fb446a6a20 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:22.243324 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:22.266236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28aea57-6b50-4261-b140-4b4e1f2363e1 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:22.282916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963e5f4-05d0-4ad7-ac4b-09fb446a6a20 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:22.305911 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974e5a4-ac2f-427f-aa2f-e17574356312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:22.306930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974e5a4-ac2f-427f-aa2f-e17574356312 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:22.307487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974e5a4-ac2f-427f-aa2f-e17574356312 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:22.307925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974e5a4-ac2f-427f-aa2f-e17574356312 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:22.308818 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974e5a4-ac2f-427f-aa2f-e17574356312 None admin] RBAC: Adding query filter params (type=network) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:22.309291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974e5a4-ac2f-427f-aa2f-e17574356312 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:22.311179 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974e5a4-ac2f-427f-aa2f-e17574356312 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:22.317525 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 383/778] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Jul 12 21:53:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:22.318531 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974e5a4-ac2f-427f-aa2f-e17574356312 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:22.323758 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7d533-ee10-4ab4-a9de-81d877856801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:22.324438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7d533-ee10-4ab4-a9de-81d877856801 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:22.324737 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7d533-ee10-4ab4-a9de-81d877856801 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:22.325031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7d533-ee10-4ab4-a9de-81d877856801 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:22.325835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7d533-ee10-4ab4-a9de-81d877856801 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:22.326148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7d533-ee10-4ab4-a9de-81d877856801 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:22.327591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7d533-ee10-4ab4-a9de-81d877856801 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:22.342254 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 396/779] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:22.343095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7d533-ee10-4ab4-a9de-81d877856801 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:22.375013 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:22.431019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974e5a4-ac2f-427f-aa2f-e17574356312 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:22.453529 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7d533-ee10-4ab4-a9de-81d877856801 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:22.470093 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6634e-abba-4c44-b875-61ea8ed8d06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:22.471279 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6634e-abba-4c44-b875-61ea8ed8d06e None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:22.471777 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6634e-abba-4c44-b875-61ea8ed8d06e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:22.472311 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6634e-abba-4c44-b875-61ea8ed8d06e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:22.473174 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6634e-abba-4c44-b875-61ea8ed8d06e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:22.474981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6634e-abba-4c44-b875-61ea8ed8d06e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:22.483760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:22.490197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-efb6634e-abba-4c44-b875-61ea8ed8d06e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 21:53:22.490197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:22.490197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:22.490197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:22.490197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:22.490197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:22.490197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:22.490197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:22.490197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 21:53:22.490197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:22.490197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:22.490197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:22.490197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:22.490197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:22.490197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:22.490197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:22.491140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:22.491140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:22.491140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:22.491140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:22.491140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:22.491140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:22.491140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:22.491140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:22.491140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:22.491140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:22.491140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:22.491140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:22.491140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:22.491140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:22.491140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:22.491140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:22.492517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:22.492517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:22.492517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:22.492517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:22.492517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:22.492517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:22.492517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:22.492517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:22.492517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:22.492517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:22.492517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:22.492517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:22.492517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:22.492517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 21:53:22.492517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:22.493330 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4aa8b-789f-49b2-a9f7-cb9e690695e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:22.493330 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4aa8b-789f-49b2-a9f7-cb9e690695e5 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:22.493330 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4aa8b-789f-49b2-a9f7-cb9e690695e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:22.493330 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4aa8b-789f-49b2-a9f7-cb9e690695e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:22.493590 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4aa8b-789f-49b2-a9f7-cb9e690695e5 None admin] RBAC: Adding query filter params (interface=public, service_id=825d49fafbc9414baaf6ccd7cb553f97, region_id=RegionOne) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:22.493777 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4aa8b-789f-49b2-a9f7-cb9e690695e5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:22.495093 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4aa8b-789f-49b2-a9f7-cb9e690695e5 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:22.495689 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 384/780] 10.222.0.44 () {56 vars in 1129 bytes} [Tue Jul 12 21:53:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:22.496606 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6634e-abba-4c44-b875-61ea8ed8d06e None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:22.512574 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 397/781] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Jul 12 21:53:22 2022] GET /identity/v3/endpoints?service_id=825d49fafbc9414baaf6ccd7cb553f97&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:22.513606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4aa8b-789f-49b2-a9f7-cb9e690695e5 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:22.612444 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6634e-abba-4c44-b875-61ea8ed8d06e None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:22.626846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4aa8b-789f-49b2-a9f7-cb9e690695e5 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:22.629041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:22.653163 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce99546-db6c-4e31-9247-345fea3ea585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:22.654934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce99546-db6c-4e31-9247-345fea3ea585 None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:22.655569 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce99546-db6c-4e31-9247-345fea3ea585 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:22.655963 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce99546-db6c-4e31-9247-345fea3ea585 None admin] PATH_INFO: `/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/24915625354f418eb7bc4d5ada85fdb0/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:22.659123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce99546-db6c-4e31-9247-345fea3ea585 None admin] RBAC: Authorizing `identity:create_grant(project_id=459e4743ff3043f69408ca2095be3bd0, user_id=24915625354f418eb7bc4d5ada85fdb0, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:22.661784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce99546-db6c-4e31-9247-345fea3ea585 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:22.665772 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da2afa-c28d-4d04-9920-e566715d5144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:22.666606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da2afa-c28d-4d04-9920-e566715d5144 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:22.666936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da2afa-c28d-4d04-9920-e566715d5144 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:22.667240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da2afa-c28d-4d04-9920-e566715d5144 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:22.668019 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da2afa-c28d-4d04-9920-e566715d5144 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:22.668366 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da2afa-c28d-4d04-9920-e566715d5144 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:22.669926 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da2afa-c28d-4d04-9920-e566715d5144 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:22.681233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 385/783] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:53:22 2022] PUT /identity/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/24915625354f418eb7bc4d5ada85fdb0/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:53:22.681958 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 398/783] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Jul 12 21:53:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:22.682612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce99546-db6c-4e31-9247-345fea3ea585 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:22.683048 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da2afa-c28d-4d04-9920-e566715d5144 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:22.752408 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:22.806282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da2afa-c28d-4d04-9920-e566715d5144 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:22.845521 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0ed90-4a84-47a5-87b0-a2e04bf2d9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:22.846225 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0ed90-4a84-47a5-87b0-a2e04bf2d9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:22.846514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0ed90-4a84-47a5-87b0-a2e04bf2d9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:22.846796 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0ed90-4a84-47a5-87b0-a2e04bf2d9e9 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:22.847532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0ed90-4a84-47a5-87b0-a2e04bf2d9e9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:22.847848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0ed90-4a84-47a5-87b0-a2e04bf2d9e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:22.849392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0ed90-4a84-47a5-87b0-a2e04bf2d9e9 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:22.857504 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 399/784] 10.222.0.44 () {56 vars in 1124 bytes} [Tue Jul 12 21:53:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:22.870076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0ed90-4a84-47a5-87b0-a2e04bf2d9e9 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:22.885603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce99546-db6c-4e31-9247-345fea3ea585 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:22.957823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f61f83-756b-493f-866a-a165a55a9113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:22.958611 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f61f83-756b-493f-866a-a165a55a9113 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:22.959047 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f61f83-756b-493f-866a-a165a55a9113 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:22.959334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f61f83-756b-493f-866a-a165a55a9113 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:22.960033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f61f83-756b-493f-866a-a165a55a9113 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:22.962667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f61f83-756b-493f-866a-a165a55a9113 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:22.977056 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 386/785] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Jul 12 21:53:22 2022] GET /identity/v3/services => generated 772 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:23.042327 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0ed90-4a84-47a5-87b0-a2e04bf2d9e9 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:23.059226 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02777e62-6a1d-4a55-936a-af6f1278c348 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:23.059563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02777e62-6a1d-4a55-936a-af6f1278c348 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:23.059802 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02777e62-6a1d-4a55-936a-af6f1278c348 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:23.061207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 387/786] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:53:23.072570 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:23.074904 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc29d7-67ca-43a6-bc12-6b0a11034ad8 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:23.075348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc29d7-67ca-43a6-bc12-6b0a11034ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:23.075696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc29d7-67ca-43a6-bc12-6b0a11034ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:23.082616 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb14152-96d0-4916-9e67-abd3bb3032e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:23.083892 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb14152-96d0-4916-9e67-abd3bb3032e6 None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:23.084176 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb14152-96d0-4916-9e67-abd3bb3032e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:23.084514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb14152-96d0-4916-9e67-abd3bb3032e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:23.085492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb14152-96d0-4916-9e67-abd3bb3032e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:23.086904 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb14152-96d0-4916-9e67-abd3bb3032e6 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:23.108980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-e4bc29d7-67ca-43a6-bc12-6b0a11034ad8 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:23.110218 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e4bc29d7-67ca-43a6-bc12-6b0a11034ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:23.127252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 400/787] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Jul 12 21:53:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:53:23.172058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:23.241099 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 388/788] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:53:23.266111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb14152-96d0-4916-9e67-abd3bb3032e6 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:23.380369 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb14152-96d0-4916-9e67-abd3bb3032e6 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:23.417693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127c8da-9e6a-412c-854d-13bb02729bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:23.418497 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127c8da-9e6a-412c-854d-13bb02729bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:23.418782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127c8da-9e6a-412c-854d-13bb02729bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:23.419062 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127c8da-9e6a-412c-854d-13bb02729bd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:23.430522 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127c8da-9e6a-412c-854d-13bb02729bd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:23.438363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-5127c8da-9e6a-412c-854d-13bb02729bd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:53:23.438363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:23.438363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:23.438363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:23.438363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:23.438363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:23.438363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:23.438363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:23.438363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:23.438363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:23.438363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:23.438363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:23.438363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:23.438363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:23.438363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:23.438363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:23.439275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:23.439275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:23.439275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:23.439275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:23.439275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:23.439275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:23.439275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:23.439275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:23.439275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:23.439275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:23.439275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:23.439275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:23.439275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:23.439275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:23.439275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:23.439275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:23.440111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:23.440111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:23.440111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:23.440111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:23.440111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:23.440111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:23.440111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:23.440111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:23.440111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:23.440111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:23.440111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:53:23.440111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:23.440111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 401/789] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:53:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:23.447918 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc29d7-67ca-43a6-bc12-6b0a11034ad8 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:23.567262 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc29d7-67ca-43a6-bc12-6b0a11034ad8 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:23.614290 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50db69-89a3-4545-b301-17fa0041d24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:23.614970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50db69-89a3-4545-b301-17fa0041d24d None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:23.615253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50db69-89a3-4545-b301-17fa0041d24d None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:23.615528 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50db69-89a3-4545-b301-17fa0041d24d None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:23.616284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50db69-89a3-4545-b301-17fa0041d24d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:23.617247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50db69-89a3-4545-b301-17fa0041d24d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:23.618640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50db69-89a3-4545-b301-17fa0041d24d None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:23.629163 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 389/790] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:53:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:23.635075 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127c8da-9e6a-412c-854d-13bb02729bd2 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:23.717440 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:23.795011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127c8da-9e6a-412c-854d-13bb02729bd2 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:23.838942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddc7a9-abc2-4b43-97fe-7ee80318c40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:23.839746 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddc7a9-abc2-4b43-97fe-7ee80318c40b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:23.840032 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddc7a9-abc2-4b43-97fe-7ee80318c40b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:23.840312 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddc7a9-abc2-4b43-97fe-7ee80318c40b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:23.861489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddc7a9-abc2-4b43-97fe-7ee80318c40b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:23.862970 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddc7a9-abc2-4b43-97fe-7ee80318c40b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:23.885566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-feddc7a9-abc2-4b43-97fe-7ee80318c40b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:23.885566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:23.885566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:23.885566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:23.885566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:23.885566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:23.885566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:23.885566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:23.885566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:23.885566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:23.885566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:23.885566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:23.885566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:23.885566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:23.885566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:23.886494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:23.886494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:23.886494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:23.886494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:23.886494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:23.886494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:23.886494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:23.886494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:23.886494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:23.886494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:23.886494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:23.886494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:23.886494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:23.886494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:23.886494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:23.886494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:23.886494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:23.887484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:23.887484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:23.887484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:23.887484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:23.887484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 402/791] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:23.892797 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50db69-89a3-4545-b301-17fa0041d24d None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:23.957704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:23.986047 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95514a-50a2-474b-896b-a003371431c2 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:23.986338 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95514a-50a2-474b-896b-a003371431c2 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:23.986622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95514a-50a2-474b-896b-a003371431c2 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:23.988379 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 403/792] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:23.999360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8593ac4-0db1-4ec6-9c33-6bb3b4f1ba21 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:23.999656 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8593ac4-0db1-4ec6-9c33-6bb3b4f1ba21 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:23.999940 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8593ac4-0db1-4ec6-9c33-6bb3b4f1ba21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:24.030021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-a8593ac4-0db1-4ec6-9c33-6bb3b4f1ba21 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:24.031343 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a8593ac4-0db1-4ec6-9c33-6bb3b4f1ba21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:24.064806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50db69-89a3-4545-b301-17fa0041d24d None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:24.110896 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 404/793] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:53:24.122819 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d8ec9-6ad8-48ed-9bb1-f88970bf2ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:24.123680 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d8ec9-6ad8-48ed-9bb1-f88970bf2ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:24.124130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d8ec9-6ad8-48ed-9bb1-f88970bf2ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:24.124547 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d8ec9-6ad8-48ed-9bb1-f88970bf2ca4 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:24.125435 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d8ec9-6ad8-48ed-9bb1-f88970bf2ca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:24.125883 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d8ec9-6ad8-48ed-9bb1-f88970bf2ca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:24.127391 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d8ec9-6ad8-48ed-9bb1-f88970bf2ca4 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:24.148154 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 390/794] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:24.153181 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8593ac4-0db1-4ec6-9c33-6bb3b4f1ba21 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:24.156971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d8ec9-6ad8-48ed-9bb1-f88970bf2ca4 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:24.203003 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:24.287569 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:24.313778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d8ec9-6ad8-48ed-9bb1-f88970bf2ca4 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:24.346662 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8593ac4-0db1-4ec6-9c33-6bb3b4f1ba21 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:24.378788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2a00e-0a7d-42a6-8d86-1d5085028356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:24.379595 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2a00e-0a7d-42a6-8d86-1d5085028356 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:24.379885 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2a00e-0a7d-42a6-8d86-1d5085028356 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:24.380165 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2a00e-0a7d-42a6-8d86-1d5085028356 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:24.401087 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2a00e-0a7d-42a6-8d86-1d5085028356 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:24.412902 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb02da9-850d-40a7-b130-7e609c6a3474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:24.414243 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb02da9-850d-40a7-b130-7e609c6a3474 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:24.414530 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb02da9-850d-40a7-b130-7e609c6a3474 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:24.414811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb02da9-850d-40a7-b130-7e609c6a3474 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:24.416016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-0db2a00e-0a7d-42a6-8d86-1d5085028356 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:24.416016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:24.416016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:24.416016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:24.416016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:24.416016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:24.416016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:24.416016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:24.416016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:24.416016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:24.416016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:24.416016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:24.416016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:24.416016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:24.416016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:24.416016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:24.416961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:24.416961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:24.416961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:24.416961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:24.416961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:24.416961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:24.416961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:24.416961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:24.416961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:24.416961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:24.416961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:24.416961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:24.416961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:24.416961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:24.416961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:24.416961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:24.418220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:24.418220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:24.418220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:24.418220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:24.418220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:24.418220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:24.418220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:24.418220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:24.418220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:24.418220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:24.418220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:24.418220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:24.419022 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 391/795] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:53:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:24.425403 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2a00e-0a7d-42a6-8d86-1d5085028356 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:24.433399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb02da9-850d-40a7-b130-7e609c6a3474 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:24.434880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb02da9-850d-40a7-b130-7e609c6a3474 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:24.458674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-ffb02da9-850d-40a7-b130-7e609c6a3474 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 21:53:24.458674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:24.458674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:24.458674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:24.458674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:24.458674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:24.458674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:24.458674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:24.458674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:24.458674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:24.458674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:24.458674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:24.458674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:24.458674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:24.458674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:24.458674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:24.459652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:24.461134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:24.461134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:24.461134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:24.461134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:24.461134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:24.461134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:24.461134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:24.461134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:24.461134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 21:53:24.461134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:24.461134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 405/796] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:53:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:24.473655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb02da9-850d-40a7-b130-7e609c6a3474 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:24.614593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2a00e-0a7d-42a6-8d86-1d5085028356 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:24.650967 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c9148-b0db-4454-9a27-dda39e7191ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:24.651998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c9148-b0db-4454-9a27-dda39e7191ce None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:24.652458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c9148-b0db-4454-9a27-dda39e7191ce None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:24.653006 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c9148-b0db-4454-9a27-dda39e7191ce None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:24.653541 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:24.655389 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c9148-b0db-4454-9a27-dda39e7191ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:24.655878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c9148-b0db-4454-9a27-dda39e7191ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:24.657491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c9148-b0db-4454-9a27-dda39e7191ce None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:24.668303 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 392/797] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:53:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:24.677830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c9148-b0db-4454-9a27-dda39e7191ce None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:24.682114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb02da9-850d-40a7-b130-7e609c6a3474 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:24.705164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:24.726896 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621ab0d-1421-4fc9-857d-ac0781a36f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:24.727965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621ab0d-1421-4fc9-857d-ac0781a36f6d None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:24.728471 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621ab0d-1421-4fc9-857d-ac0781a36f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:24.729027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621ab0d-1421-4fc9-857d-ac0781a36f6d None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:24.730220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621ab0d-1421-4fc9-857d-ac0781a36f6d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:24.730739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621ab0d-1421-4fc9-857d-ac0781a36f6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:24.732371 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621ab0d-1421-4fc9-857d-ac0781a36f6d None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:24.752142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 406/798] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Jul 12 21:53:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:24.757767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621ab0d-1421-4fc9-857d-ac0781a36f6d None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:24.892414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621ab0d-1421-4fc9-857d-ac0781a36f6d None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:24.933558 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c9148-b0db-4454-9a27-dda39e7191ce None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:24.975561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd7b0b-6e5f-4bb5-b13b-bf512fd42610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:24.976474 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd7b0b-6e5f-4bb5-b13b-bf512fd42610 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:24.976792 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd7b0b-6e5f-4bb5-b13b-bf512fd42610 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:24.977571 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd7b0b-6e5f-4bb5-b13b-bf512fd42610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:24.985850 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:24.992031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd7b0b-6e5f-4bb5-b13b-bf512fd42610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:24.994320 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd7b0b-6e5f-4bb5-b13b-bf512fd42610 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:25.014893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-d9fd7b0b-6e5f-4bb5-b13b-bf512fd42610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:25.014893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:25.014893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:25.014893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:25.014893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:25.014893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:25.014893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:25.014893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:25.014893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:25.014893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:25.014893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:25.014893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:25.014893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:25.014893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:25.014893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:25.015811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:25.015811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:25.015811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:25.015811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:25.015811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:25.015811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:25.015811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:25.015811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:25.015811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:25.015811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:25.015811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:25.015811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:25.015811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:25.015811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:25.015811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:25.015811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:25.015811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:25.016756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:25.016756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:25.016756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:25.016756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:25.023560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 407/799] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:25.025053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd7b0b-6e5f-4bb5-b13b-bf512fd42610 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:25.033624 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0686e65-2873-4919-a87a-de9e0a75a3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:25.038770 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0686e65-2873-4919-a87a-de9e0a75a3ce None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:25.039212 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0686e65-2873-4919-a87a-de9e0a75a3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:25.039629 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0686e65-2873-4919-a87a-de9e0a75a3ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:25.058457 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0686e65-2873-4919-a87a-de9e0a75a3ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:25.062164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0686e65-2873-4919-a87a-de9e0a75a3ce None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:25.092561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-b0686e65-2873-4919-a87a-de9e0a75a3ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:25.092561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:25.092561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:25.092561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:25.092561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:25.092561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:25.092561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:25.092561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:25.092561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:25.092561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:25.092561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:25.092561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:25.092561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:25.092561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:25.092561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:25.094204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:25.094204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:25.094204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:25.094204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:25.094204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:25.094204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:25.094204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:25.094204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:25.094204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:25.094204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:25.094204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:25.094204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:25.094204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:25.094204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:25.094204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:25.094204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:25.094204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:25.095278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:25.095278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:25.095278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:25.095278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:25.106973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 393/800] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:25.108083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0686e65-2873-4919-a87a-de9e0a75a3ce None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:25.192839 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd7b0b-6e5f-4bb5-b13b-bf512fd42610 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:25.233788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf08339-c7eb-462a-839e-6978d1c6ad2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:25.234893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf08339-c7eb-462a-839e-6978d1c6ad2b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:25.235345 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf08339-c7eb-462a-839e-6978d1c6ad2b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:25.235760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf08339-c7eb-462a-839e-6978d1c6ad2b None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:25.236681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf08339-c7eb-462a-839e-6978d1c6ad2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:25.237152 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf08339-c7eb-462a-839e-6978d1c6ad2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:25.239190 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf08339-c7eb-462a-839e-6978d1c6ad2b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:25.258776 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 408/801] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:25.260286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf08339-c7eb-462a-839e-6978d1c6ad2b None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:25.266083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:25.320755 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:25.376458 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf08339-c7eb-462a-839e-6978d1c6ad2b None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:25.399110 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0686e65-2873-4919-a87a-de9e0a75a3ce None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:25.417708 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a596b8a-e393-4d62-a1d5-c3b250022255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:25.418501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a596b8a-e393-4d62-a1d5-c3b250022255 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:25.418780 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a596b8a-e393-4d62-a1d5-c3b250022255 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:25.419054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a596b8a-e393-4d62-a1d5-c3b250022255 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:25.426746 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a596b8a-e393-4d62-a1d5-c3b250022255 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:25.428196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a596b8a-e393-4d62-a1d5-c3b250022255 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:25.439138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c0d1f-c9a4-453e-8d89-61bf0f4bfe84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:25.440028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c0d1f-c9a4-453e-8d89-61bf0f4bfe84 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:25.440473 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c0d1f-c9a4-453e-8d89-61bf0f4bfe84 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:25.440945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c0d1f-c9a4-453e-8d89-61bf0f4bfe84 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:25.442069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c0d1f-c9a4-453e-8d89-61bf0f4bfe84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:25.442529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c0d1f-c9a4-453e-8d89-61bf0f4bfe84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:25.443615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-3a596b8a-e393-4d62-a1d5-c3b250022255 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:25.443615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:25.443615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:25.443615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:25.443615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:25.443615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:25.443615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:25.443615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:25.443615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:25.443615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:25.443615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:25.443615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:25.443615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:25.443615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:25.443615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:25.444680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:25.444680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:25.444680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:25.444680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:25.444680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:25.444680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:25.444680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:25.444680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:25.444680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:25.444680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:25.444680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:25.444680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:25.444680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:25.444680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:25.444680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:25.444680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:25.444680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:25.445682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:25.445682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:25.445682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:25.445682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:25.445682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:25.445682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:25.445682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:25.445682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:25.445682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 409/802] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:25.446372 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5afb6-c969-402f-8c84-6249142d12f3 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:25.446664 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5afb6-c969-402f-8c84-6249142d12f3 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:25.446946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede5afb6-c969-402f-8c84-6249142d12f3 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:25.447149 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c0d1f-c9a4-453e-8d89-61bf0f4bfe84 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:25.448837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 410/803] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:25.450119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5c58f-0c95-42af-a382-9007b684acb9 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:25.450571 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5c58f-0c95-42af-a382-9007b684acb9 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:25.451846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5c58f-0c95-42af-a382-9007b684acb9 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:25.453302 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 411/804] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:25.454156 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5c58f-0c95-42af-a382-9007b684acb9 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:25.464459 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 394/805] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:25.466426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825f102-5a37-469e-a9d8-a0f520c39c28 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:25.466815 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825f102-5a37-469e-a9d8-a0f520c39c28 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:25.467189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825f102-5a37-469e-a9d8-a0f520c39c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:25.485888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-6825f102-5a37-469e-a9d8-a0f520c39c28 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:25.487160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6825f102-5a37-469e-a9d8-a0f520c39c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:25.561181 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 395/806] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:25.562615 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0f670-9dc4-43a2-b6db-1dad52c04bba None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:25.562967 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0f670-9dc4-43a2-b6db-1dad52c04bba None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:25.563319 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0f670-9dc4-43a2-b6db-1dad52c04bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:25.574373 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5c58f-0c95-42af-a382-9007b684acb9 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:25.589277 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-25c0f670-9dc4-43a2-b6db-1dad52c04bba None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:25.589532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:25.590637 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-25c0f670-9dc4-43a2-b6db-1dad52c04bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:25.618792 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16384afd-2efc-4cdb-bd23-f596bbb286a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:25.619472 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16384afd-2efc-4cdb-bd23-f596bbb286a2 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:25.619936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16384afd-2efc-4cdb-bd23-f596bbb286a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:25.620045 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16384afd-2efc-4cdb-bd23-f596bbb286a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:25.620832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16384afd-2efc-4cdb-bd23-f596bbb286a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:25.621151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16384afd-2efc-4cdb-bd23-f596bbb286a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:25.622609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16384afd-2efc-4cdb-bd23-f596bbb286a2 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:25.636805 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 412/807] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:25.642056 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:25.642283 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16384afd-2efc-4cdb-bd23-f596bbb286a2 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:25.667311 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 396/808] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:25.668487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0f670-9dc4-43a2-b6db-1dad52c04bba None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:25.761212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16384afd-2efc-4cdb-bd23-f596bbb286a2 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:25.780606 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0f670-9dc4-43a2-b6db-1dad52c04bba None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:25.795892 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b1cd5-efd0-42c9-b4bf-32d0c89987f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:25.796403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b1cd5-efd0-42c9-b4bf-32d0c89987f2 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:25.796693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b1cd5-efd0-42c9-b4bf-32d0c89987f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:25.797000 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b1cd5-efd0-42c9-b4bf-32d0c89987f2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:25.805388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929b1cd5-efd0-42c9-b4bf-32d0c89987f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:25.806877 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929b1cd5-efd0-42c9-b4bf-32d0c89987f2 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:25.815304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577f284-80c0-4105-a9d7-32453f3b3475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:25.816099 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577f284-80c0-4105-a9d7-32453f3b3475 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:25.816383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577f284-80c0-4105-a9d7-32453f3b3475 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:25.816944 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577f284-80c0-4105-a9d7-32453f3b3475 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:25.821976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-929b1cd5-efd0-42c9-b4bf-32d0c89987f2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 21:53:25.821976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:25.821976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:25.821976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:25.821976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:25.821976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:25.821976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:25.821976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:25.821976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:25.821976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:25.821976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:25.821976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:25.821976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:25.821976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:25.821976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:25.821976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:25.822957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:25.823887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:25.823887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:25.823887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:25.823887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:25.823887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:25.823887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:25.823887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:25.823887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:25.823887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 21:53:25.823887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:25.823887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 413/809] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Jul 12 21:53:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:25.825047 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b1cd5-efd0-42c9-b4bf-32d0c89987f2 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:25.831803 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577f284-80c0-4105-a9d7-32453f3b3475 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:25.840366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-d577f284-80c0-4105-a9d7-32453f3b3475 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:53:25.840366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:25.840366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:25.840366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:25.840366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:25.840366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:25.840366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:25.840366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:25.840366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:25.840366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:25.840366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:25.840366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:25.840366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:25.840366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:25.840366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:25.840366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:25.841358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:25.841358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:25.841358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:25.841358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:25.841358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:25.841358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:25.841358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:25.841358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:25.841358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:25.841358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:25.841358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:25.841358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:25.841358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:25.841358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:25.841358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:25.841358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:25.842207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:25.842207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:25.842207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:25.842207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:25.842207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:25.842207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:25.842207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:25.842207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:25.842207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:25.842207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:25.842207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:53:25.842207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:25.843025 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 397/810] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:53:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:25.843981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577f284-80c0-4105-a9d7-32453f3b3475 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:25.855926 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:25.890169 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:25.928947 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b1cd5-efd0-42c9-b4bf-32d0c89987f2 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:25.951884 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577f284-80c0-4105-a9d7-32453f3b3475 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:25.966875 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d19ce-6cbf-45fc-8486-dad0f35ef6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:25.967783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d19ce-6cbf-45fc-8486-dad0f35ef6f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:25.968068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d19ce-6cbf-45fc-8486-dad0f35ef6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:25.968344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d19ce-6cbf-45fc-8486-dad0f35ef6f0 None admin] PATH_INFO: `/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/837bb5ac18554dfdbbcc776689639494/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:25.970366 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d19ce-6cbf-45fc-8486-dad0f35ef6f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=459e4743ff3043f69408ca2095be3bd0, user_id=837bb5ac18554dfdbbcc776689639494, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:25.971790 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d19ce-6cbf-45fc-8486-dad0f35ef6f0 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:25.985197 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 414/811] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:53:25 2022] PUT /identity/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/837bb5ac18554dfdbbcc776689639494/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:53:25.986575 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc53323-b020-4612-a8de-59ba852ac834 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:25.986874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc53323-b020-4612-a8de-59ba852ac834 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:25.987161 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc53323-b020-4612-a8de-59ba852ac834 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:25.987466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3c939-7510-4cf2-a4de-988c02d143db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:25.988276 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3c939-7510-4cf2-a4de-988c02d143db None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:25.992839 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3c939-7510-4cf2-a4de-988c02d143db None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:25.992839 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3c939-7510-4cf2-a4de-988c02d143db None admin] PATH_INFO: `/v3/services/network` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:25.993100 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 415/812] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:25.993178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3c939-7510-4cf2-a4de-988c02d143db None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:25.993248 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc53323-b020-4612-a8de-59ba852ac834 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:25.993350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3c939-7510-4cf2-a4de-988c02d143db None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:26.012771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-97a3c939-7510-4cf2-a4de-988c02d143db None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 21:53:26.012771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:26.012771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:26.012771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:26.012771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:26.012771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:26.012771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:26.012771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:26.012771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 21:53:26.012771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:26.012771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:26.012771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:26.012771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:26.012771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:26.012771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:26.012771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:26.012771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:26.013851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:26.013851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:26.013851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:26.013851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:26.013851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:26.013851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:26.013851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:26.013851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:26.013851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:26.013851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:26.013851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:26.013851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:26.013851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:26.013851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:26.013851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:26.013851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:26.014730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:26.014730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:26.014730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:26.014730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:26.014730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:26.014730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:26.014730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:26.014730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:26.014730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:26.014730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:26.014730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:26.014730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:26.014730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 21:53:26.014730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:26.014730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 398/813] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:25 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:26.015557 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3c939-7510-4cf2-a4de-988c02d143db None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:26.112563 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:26.127077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc53323-b020-4612-a8de-59ba852ac834 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:26.136598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3c939-7510-4cf2-a4de-988c02d143db None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:26.139751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:26.171522 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b41616-eb80-47e9-befd-a2baa99024eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:26.172235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b41616-eb80-47e9-befd-a2baa99024eb None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:26.172477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b41616-eb80-47e9-befd-a2baa99024eb None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:26.172787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b41616-eb80-47e9-befd-a2baa99024eb None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:26.173636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b41616-eb80-47e9-befd-a2baa99024eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:26.173965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b41616-eb80-47e9-befd-a2baa99024eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:26.175384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b41616-eb80-47e9-befd-a2baa99024eb None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:26.176876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4ea49-1383-4778-b0fb-85a36124ac7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:26.177729 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4ea49-1383-4778-b0fb-85a36124ac7a None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:26.178180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4ea49-1383-4778-b0fb-85a36124ac7a None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:26.178614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4ea49-1383-4778-b0fb-85a36124ac7a None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:26.180024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4ea49-1383-4778-b0fb-85a36124ac7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:26.180251 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4ea49-1383-4778-b0fb-85a36124ac7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:26.181598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4ea49-1383-4778-b0fb-85a36124ac7a None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:26.192875 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 399/814] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:53:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:26.192875 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da5247-22a9-4ea6-88a9-11204b000f5a None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:26.192875 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da5247-22a9-4ea6-88a9-11204b000f5a None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:26.193200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da5247-22a9-4ea6-88a9-11204b000f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:26.196515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 416/815] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Jul 12 21:53:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:26.197413 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b41616-eb80-47e9-befd-a2baa99024eb None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:26.212418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-d9da5247-22a9-4ea6-88a9-11204b000f5a None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:26.213682 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d9da5247-22a9-4ea6-88a9-11204b000f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:26.289150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 400/816] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:26.290514 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da5247-22a9-4ea6-88a9-11204b000f5a None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:26.318658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b41616-eb80-47e9-befd-a2baa99024eb None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:26.357042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab376a2-fd21-4806-8272-53c6f6bf11d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:26.357708 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab376a2-fd21-4806-8272-53c6f6bf11d3 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:26.358424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab376a2-fd21-4806-8272-53c6f6bf11d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:26.358709 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab376a2-fd21-4806-8272-53c6f6bf11d3 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:26.359425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab376a2-fd21-4806-8272-53c6f6bf11d3 None admin] RBAC: Adding query filter params (name=network) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:26.359738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab376a2-fd21-4806-8272-53c6f6bf11d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:26.361144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab376a2-fd21-4806-8272-53c6f6bf11d3 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:26.370725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 417/817] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Jul 12 21:53:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:26.370725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab376a2-fd21-4806-8272-53c6f6bf11d3 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:26.384943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:26.408254 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:26.414859 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da5247-22a9-4ea6-88a9-11204b000f5a None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:26.452037 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c6ba4-d1e3-4711-ac06-c001ce306db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:26.452876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c6ba4-d1e3-4711-ac06-c001ce306db9 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:26.453162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c6ba4-d1e3-4711-ac06-c001ce306db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:26.453441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c6ba4-d1e3-4711-ac06-c001ce306db9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:26.464931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c6ba4-d1e3-4711-ac06-c001ce306db9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:26.466396 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c6ba4-d1e3-4711-ac06-c001ce306db9 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:26.484026 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab376a2-fd21-4806-8272-53c6f6bf11d3 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:26.484327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-c99c6ba4-d1e3-4711-ac06-c001ce306db9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:26.484327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:26.484327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:26.484327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:26.484327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:26.484327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:26.484327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:26.484327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:26.484327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:26.484327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:26.484327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:26.484327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:26.484327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:26.484327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:26.484327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:26.485253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:26.485253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:26.485253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:26.485253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:26.485253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:26.485253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:26.485253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:26.485253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:26.485253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:26.485253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:26.485253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:26.485253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:26.485253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:26.485253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:26.485253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:26.485253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:26.485253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:26.486700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:26.486700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:26.486700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:26.486700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:26.486700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 401/818] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:26.486700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c6ba4-d1e3-4711-ac06-c001ce306db9 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:26.523357 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21d6b9-1955-4129-b20c-ff9c2f4e4e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:26.524321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21d6b9-1955-4129-b20c-ff9c2f4e4e56 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:26.524503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21d6b9-1955-4129-b20c-ff9c2f4e4e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:26.524806 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21d6b9-1955-4129-b20c-ff9c2f4e4e56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:26.535009 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21d6b9-1955-4129-b20c-ff9c2f4e4e56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:26.536470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21d6b9-1955-4129-b20c-ff9c2f4e4e56 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:26.555386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-7b21d6b9-1955-4129-b20c-ff9c2f4e4e56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:26.555386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:26.555386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:26.555386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:26.555386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:26.555386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:26.555386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:26.555386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:26.555386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:26.555386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:26.555386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:26.555386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:26.555386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:26.555386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:26.555386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:26.556281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:26.556281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:26.556281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:26.556281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:26.556281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:26.556281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:26.556281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:26.556281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:26.556281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:26.556281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:26.556281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:26.556281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:26.556281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:26.556281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:26.556281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:26.556281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:26.556281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:26.557236 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:26.557236 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:26.557236 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:26.557236 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:26.561220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 418/819] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:26.561803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21d6b9-1955-4129-b20c-ff9c2f4e4e56 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:26.604452 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c6ba4-d1e3-4711-ac06-c001ce306db9 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:26.630592 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:26.648583 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6b742-0887-492a-b269-41da84c01a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:26.649213 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6b742-0887-492a-b269-41da84c01a15 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:26.649558 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6b742-0887-492a-b269-41da84c01a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:26.649558 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6b742-0887-492a-b269-41da84c01a15 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:26.650383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6b742-0887-492a-b269-41da84c01a15 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:26.651798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6b742-0887-492a-b269-41da84c01a15 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:26.666631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-c6c6b742-0887-492a-b269-41da84c01a15 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 21:53:26.666631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:26.666631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:26.666631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:26.666631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:26.666631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:26.666631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:26.666631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:26.666631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 21:53:26.666631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:26.666631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:26.666631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:26.666631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:26.666631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:26.666631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:26.666631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:26.666631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:26.667701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:26.667701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:26.667701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:26.667701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:26.667701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:26.667701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:26.667701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:26.667701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:26.667701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:26.667701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:26.667701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:26.667701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:26.667701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:26.667701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:26.667701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:26.667701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:26.669572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:26.669572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:26.669572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:26.669572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:26.669572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:26.669572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:26.669572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:26.669572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:26.669572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:26.669572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:26.669572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:26.669572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:26.669572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 21:53:26.669572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:26.669572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 402/820] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:53:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:26.670472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6b742-0887-492a-b269-41da84c01a15 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:26.686567 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21d6b9-1955-4129-b20c-ff9c2f4e4e56 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:26.691199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:26.727622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ed27d-a31d-40d0-b92a-00a5ab7f6b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:26.728452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ed27d-a31d-40d0-b92a-00a5ab7f6b83 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:26.728761 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ed27d-a31d-40d0-b92a-00a5ab7f6b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:26.729046 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ed27d-a31d-40d0-b92a-00a5ab7f6b83 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:26.729758 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ed27d-a31d-40d0-b92a-00a5ab7f6b83 None admin] RBAC: Adding query filter params (type=network) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:26.730068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ed27d-a31d-40d0-b92a-00a5ab7f6b83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:26.731481 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ed27d-a31d-40d0-b92a-00a5ab7f6b83 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:26.737076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 419/821] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Jul 12 21:53:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:26.737742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ed27d-a31d-40d0-b92a-00a5ab7f6b83 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:26.798431 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6b742-0887-492a-b269-41da84c01a15 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:26.840274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68feb3-0d36-4148-aaac-387754f0c314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:26.840900 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68feb3-0d36-4148-aaac-387754f0c314 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:26.841189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68feb3-0d36-4148-aaac-387754f0c314 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:26.841490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68feb3-0d36-4148-aaac-387754f0c314 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:26.842211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa68feb3-0d36-4148-aaac-387754f0c314 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:26.842517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa68feb3-0d36-4148-aaac-387754f0c314 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:26.844124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa68feb3-0d36-4148-aaac-387754f0c314 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:26.849171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ed27d-a31d-40d0-b92a-00a5ab7f6b83 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:26.862689 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 403/822] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:26.863862 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68feb3-0d36-4148-aaac-387754f0c314 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:26.886853 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:26.897117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a6eaa-67f9-47ed-a7ff-ec59815fcc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:26.898188 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a6eaa-67f9-47ed-a7ff-ec59815fcc32 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:26.898596 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a6eaa-67f9-47ed-a7ff-ec59815fcc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:26.899046 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a6eaa-67f9-47ed-a7ff-ec59815fcc32 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:26.899606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4a6eaa-67f9-47ed-a7ff-ec59815fcc32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:26.899921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4a6eaa-67f9-47ed-a7ff-ec59815fcc32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:26.901322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4a6eaa-67f9-47ed-a7ff-ec59815fcc32 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:26.914254 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 420/823] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:26.919358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a6eaa-67f9-47ed-a7ff-ec59815fcc32 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:26.942751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:26.976703 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68feb3-0d36-4148-aaac-387754f0c314 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:27.013259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f5bd1-6799-428c-b859-470dccb94896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:27.013931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f5bd1-6799-428c-b859-470dccb94896 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:27.014219 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f5bd1-6799-428c-b859-470dccb94896 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:27.014508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f5bd1-6799-428c-b859-470dccb94896 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:27.015217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f5bd1-6799-428c-b859-470dccb94896 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:27.015677 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f5bd1-6799-428c-b859-470dccb94896 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:27.016940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f5bd1-6799-428c-b859-470dccb94896 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:27.022307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 404/824] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:53:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:27.023899 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f5bd1-6799-428c-b859-470dccb94896 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:27.039283 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a6eaa-67f9-47ed-a7ff-ec59815fcc32 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:27.079823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a5208-d017-4fc2-9761-6283ef026008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:27.080487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a5208-d017-4fc2-9761-6283ef026008 None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:27.080803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a5208-d017-4fc2-9761-6283ef026008 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:27.081088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a5208-d017-4fc2-9761-6283ef026008 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:27.082299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a5208-d017-4fc2-9761-6283ef026008 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:27.083736 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a5208-d017-4fc2-9761-6283ef026008 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:27.116893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 421/825] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Jul 12 21:53:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:53:27.116893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a5208-d017-4fc2-9761-6283ef026008 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:27.150089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f5bd1-6799-428c-b859-470dccb94896 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:27.167385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:27.200181 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:27.208540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c9926-a6d5-4655-be32-ee2a2ad1258e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:27.209611 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c9926-a6d5-4655-be32-ee2a2ad1258e None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:27.210054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c9926-a6d5-4655-be32-ee2a2ad1258e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:27.210432 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c9926-a6d5-4655-be32-ee2a2ad1258e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:27.221587 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c9926-a6d5-4655-be32-ee2a2ad1258e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:27.224335 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c9926-a6d5-4655-be32-ee2a2ad1258e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:27.245957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-340c9926-a6d5-4655-be32-ee2a2ad1258e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:53:27.245957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:27.245957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:27.245957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:27.245957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:27.245957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:27.245957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:27.245957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:27.245957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:27.245957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:27.245957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:27.245957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:27.245957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:27.245957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:27.245957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:27.245957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:27.247164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:27.248201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:27.248201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:27.248201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:27.248201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:27.248201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:27.248201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:27.248201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:27.248201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:27.248201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:53:27.248201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:27.248201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 405/826] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Jul 12 21:53:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:27.248970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c9926-a6d5-4655-be32-ee2a2ad1258e None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:27.253645 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a5208-d017-4fc2-9761-6283ef026008 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:27.307208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33da46b-c09a-415d-8b1b-5f5e1036c176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:27.308271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33da46b-c09a-415d-8b1b-5f5e1036c176 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:27.308554 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33da46b-c09a-415d-8b1b-5f5e1036c176 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:27.308858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33da46b-c09a-415d-8b1b-5f5e1036c176 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:27.317300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33da46b-c09a-415d-8b1b-5f5e1036c176 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:27.318108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33da46b-c09a-415d-8b1b-5f5e1036c176 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:27.333550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-e33da46b-c09a-415d-8b1b-5f5e1036c176 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:27.333550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:27.333550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:27.333550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:27.333550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:27.333550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:27.333550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:27.333550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:27.333550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:27.333550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:27.333550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:27.333550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:27.333550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:27.333550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:27.333550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:27.334492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:27.334492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:27.334492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:27.334492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:27.334492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:27.334492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:27.334492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:27.334492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:27.334492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:27.334492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:27.334492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:27.334492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:27.334492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:27.334492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:27.334492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:27.334492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:27.334492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:27.336252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:27.336252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:27.336252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:27.336252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:27.336252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:27.336252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:27.336252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:27.336252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:27.336252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 422/827] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:27.336252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33da46b-c09a-415d-8b1b-5f5e1036c176 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:27.395004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c9926-a6d5-4655-be32-ee2a2ad1258e None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:27.434054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0278ff-ecf5-4724-b610-bec2993752b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:27.435299 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0278ff-ecf5-4724-b610-bec2993752b6 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:27.435683 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0278ff-ecf5-4724-b610-bec2993752b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:27.436060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0278ff-ecf5-4724-b610-bec2993752b6 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:27.437258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0278ff-ecf5-4724-b610-bec2993752b6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:27.437690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0278ff-ecf5-4724-b610-bec2993752b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:27.439489 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0278ff-ecf5-4724-b610-bec2993752b6 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:27.452406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 406/828] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:53:27 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:27.452406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0278ff-ecf5-4724-b610-bec2993752b6 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:27.471275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33da46b-c09a-415d-8b1b-5f5e1036c176 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:27.475126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:27.519284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6fcc8-890f-4066-8edc-b09bfc50e557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:27.520270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6fcc8-890f-4066-8edc-b09bfc50e557 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:27.520827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6fcc8-890f-4066-8edc-b09bfc50e557 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:27.521264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6fcc8-890f-4066-8edc-b09bfc50e557 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:27.522628 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6fcc8-890f-4066-8edc-b09bfc50e557 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:27.523282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6fcc8-890f-4066-8edc-b09bfc50e557 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:27.525391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6fcc8-890f-4066-8edc-b09bfc50e557 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:27.534545 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:27.550880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 423/829] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Jul 12 21:53:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:27.561794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6fcc8-890f-4066-8edc-b09bfc50e557 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:27.588657 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0278ff-ecf5-4724-b610-bec2993752b6 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:27.629797 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03ce91-73b2-43a6-8aa3-f523902a541c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:27.630666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03ce91-73b2-43a6-8aa3-f523902a541c None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:27.631047 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03ce91-73b2-43a6-8aa3-f523902a541c None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:27.631333 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03ce91-73b2-43a6-8aa3-f523902a541c None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:27.632795 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03ce91-73b2-43a6-8aa3-f523902a541c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:27.633109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03ce91-73b2-43a6-8aa3-f523902a541c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:27.634589 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03ce91-73b2-43a6-8aa3-f523902a541c None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:27.652363 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 407/830] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:27.658679 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03ce91-73b2-43a6-8aa3-f523902a541c None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:27.690020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6fcc8-890f-4066-8edc-b09bfc50e557 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:27.730659 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07044bd-35fb-400a-a58c-bc03388c5ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:27.731344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07044bd-35fb-400a-a58c-bc03388c5ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:27.731943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07044bd-35fb-400a-a58c-bc03388c5ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:27.732253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07044bd-35fb-400a-a58c-bc03388c5ce1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:27.771513 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:27.792799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03ce91-73b2-43a6-8aa3-f523902a541c None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:27.817669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07044bd-35fb-400a-a58c-bc03388c5ce1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:27.820212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07044bd-35fb-400a-a58c-bc03388c5ce1 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:27.821418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:27.850144 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfb8dc-d036-4503-ade3-53f067fde9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:27.851347 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfb8dc-d036-4503-ade3-53f067fde9d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:27.852173 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfb8dc-d036-4503-ade3-53f067fde9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:27.852466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfb8dc-d036-4503-ade3-53f067fde9d3 None admin] PATH_INFO: `/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/8d6b823cc7dc434986f77739fa0259ea/roles/f1e42ea5a1c440bba915b459d2744858` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:27.876681 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfb8dc-d036-4503-ade3-53f067fde9d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=459e4743ff3043f69408ca2095be3bd0, user_id=8d6b823cc7dc434986f77739fa0259ea, role_id=f1e42ea5a1c440bba915b459d2744858)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:27.878119 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfb8dc-d036-4503-ade3-53f067fde9d3 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:27.900283 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 408/831] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:53:27 2022] PUT /identity/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/8d6b823cc7dc434986f77739fa0259ea/roles/f1e42ea5a1c440bba915b459d2744858 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:53:27.966506 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 424/832] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:53:27 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:53:27.973904 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfb8dc-d036-4503-ade3-53f067fde9d3 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:28.039200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ec54f-d08b-4706-b288-fabc56aa850a None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:28.039898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ec54f-d08b-4706-b288-fabc56aa850a None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:28.040408 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ec54f-d08b-4706-b288-fabc56aa850a None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:28.042247 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 425/833] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:53:28.048598 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94535e-19db-4e95-88b8-cee4d758678f None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:28.049232 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94535e-19db-4e95-88b8-cee4d758678f None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:28.049578 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94535e-19db-4e95-88b8-cee4d758678f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:28.077824 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-3b94535e-19db-4e95-88b8-cee4d758678f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:28.079161 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-3b94535e-19db-4e95-88b8-cee4d758678f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:28.098799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfb8dc-d036-4503-ade3-53f067fde9d3 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:28.141780 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af738af1-3b7c-4b51-a889-c01b472b4ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:28.142812 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:28.143513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af738af1-3b7c-4b51-a889-c01b472b4ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:28.143905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af738af1-3b7c-4b51-a889-c01b472b4ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:28.144181 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af738af1-3b7c-4b51-a889-c01b472b4ca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:28.155478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 426/834] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jul 12 21:53:28.157800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af738af1-3b7c-4b51-a889-c01b472b4ca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:28.159284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af738af1-3b7c-4b51-a889-c01b472b4ca7 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:28.178612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94535e-19db-4e95-88b8-cee4d758678f None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:28.182620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-af738af1-3b7c-4b51-a889-c01b472b4ca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:28.182620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:28.182620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:28.182620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:28.182620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:28.182620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:28.182620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:28.182620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:28.182620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:28.182620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:28.182620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:28.182620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:28.182620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:28.182620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:28.182620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:28.183496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:28.183496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:28.183496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:28.183496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:28.183496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:28.183496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:28.183496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:28.183496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:28.183496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:28.183496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:28.183496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:28.183496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:28.183496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:28.183496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:28.183496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:28.183496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:28.183496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:28.184375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:28.184375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:28.184375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:28.184375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:28.184375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 409/835] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:28.189673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af738af1-3b7c-4b51-a889-c01b472b4ca7 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:28.221238 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:28.315041 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94535e-19db-4e95-88b8-cee4d758678f None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:28.327267 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af738af1-3b7c-4b51-a889-c01b472b4ca7 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:28.353857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc90b9a-56e8-47f4-a802-c47c652d753c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:28.354795 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc90b9a-56e8-47f4-a802-c47c652d753c None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:28.355153 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc90b9a-56e8-47f4-a802-c47c652d753c None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:28.355850 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc90b9a-56e8-47f4-a802-c47c652d753c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:28.366720 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc90b9a-56e8-47f4-a802-c47c652d753c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:28.371844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3832f-e05b-429e-b1fb-7ee0e92d59d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:28.372735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3832f-e05b-429e-b1fb-7ee0e92d59d6 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:28.373216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3832f-e05b-429e-b1fb-7ee0e92d59d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:28.373664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3832f-e05b-429e-b1fb-7ee0e92d59d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:28.374556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3832f-e05b-429e-b1fb-7ee0e92d59d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:28.375039 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3832f-e05b-429e-b1fb-7ee0e92d59d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:28.375143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-1bc90b9a-56e8-47f4-a802-c47c652d753c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:53:28.375143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:28.375143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:28.375143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:28.375143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:28.375143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:28.375143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:28.375143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:28.375143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:28.375143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:28.375143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:28.375143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:28.375143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:28.375143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:28.375143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:28.375143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:28.376006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:28.376006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:28.376006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:28.376006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:28.376006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:28.376006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:28.376006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:28.376006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:28.376006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:28.376006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:28.376006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:28.376006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:28.376006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:28.376006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:28.376006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:28.376006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:28.376874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:28.376874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:28.376874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:28.376874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:28.376874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:28.376874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:28.376874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:28.376874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:28.376874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:28.376874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:28.376874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:53:28.376874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:28.379268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3832f-e05b-429e-b1fb-7ee0e92d59d6 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:28.381135 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 427/836] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:53:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:28.383381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc90b9a-56e8-47f4-a802-c47c652d753c None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:28.394740 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 410/837] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:28.401755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3832f-e05b-429e-b1fb-7ee0e92d59d6 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:28.448921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:28.506216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:28.506547 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc90b9a-56e8-47f4-a802-c47c652d753c None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:28.537043 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3832f-e05b-429e-b1fb-7ee0e92d59d6 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:28.545201 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b2a97-c000-4cae-9a83-348d60f77e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:28.545874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b2a97-c000-4cae-9a83-348d60f77e4a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:28.546156 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b2a97-c000-4cae-9a83-348d60f77e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:28.546432 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b2a97-c000-4cae-9a83-348d60f77e4a None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:28.547186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b2a97-c000-4cae-9a83-348d60f77e4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:28.547890 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b2a97-c000-4cae-9a83-348d60f77e4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:28.549122 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b2a97-c000-4cae-9a83-348d60f77e4a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:28.559285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 428/838] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:53:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:28.564380 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b2a97-c000-4cae-9a83-348d60f77e4a None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:28.581072 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bd41e-2f01-4795-9f47-fa90cca092dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:28.582188 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bd41e-2f01-4795-9f47-fa90cca092dc None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:28.583138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bd41e-2f01-4795-9f47-fa90cca092dc None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:28.583614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bd41e-2f01-4795-9f47-fa90cca092dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:28.597053 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606bd41e-2f01-4795-9f47-fa90cca092dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:28.600093 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606bd41e-2f01-4795-9f47-fa90cca092dc None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:28.623416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-606bd41e-2f01-4795-9f47-fa90cca092dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:28.623416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:28.623416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:28.623416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:28.623416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:28.623416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:28.623416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:28.623416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:28.623416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:28.623416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:28.623416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:28.623416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:28.623416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:28.623416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:28.623416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:28.624420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:28.624420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:28.624420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:28.624420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:28.624420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:28.624420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:28.624420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:28.624420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:28.624420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:28.624420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:28.624420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:28.624420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:28.624420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:28.624420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:28.624420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:28.624420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:28.624420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:28.625348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:28.625348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:28.625348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:28.625348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:28.625348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:28.625348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:28.625348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:28.625348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:28.631581 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 411/839] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:28.633808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bd41e-2f01-4795-9f47-fa90cca092dc None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:28.701467 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b2a97-c000-4cae-9a83-348d60f77e4a None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:28.717221 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:28.740602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659136e-e1bf-44a8-a3d8-37e5a41cf99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:28.742594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659136e-e1bf-44a8-a3d8-37e5a41cf99f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:28.743219 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659136e-e1bf-44a8-a3d8-37e5a41cf99f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:28.743704 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659136e-e1bf-44a8-a3d8-37e5a41cf99f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:28.755771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bd41e-2f01-4795-9f47-fa90cca092dc None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:28.761148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659136e-e1bf-44a8-a3d8-37e5a41cf99f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:28.762853 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659136e-e1bf-44a8-a3d8-37e5a41cf99f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:28.784537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-d659136e-e1bf-44a8-a3d8-37e5a41cf99f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:28.784537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:28.784537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:28.784537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:28.784537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:28.784537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:28.784537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:28.784537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:28.784537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:28.784537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:28.784537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:28.784537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:28.784537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:28.784537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:28.784537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:28.785501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:28.785501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:28.785501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:28.785501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:28.785501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:28.785501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:28.785501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:28.785501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:28.785501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:28.785501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:28.785501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:28.785501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:28.785501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:28.785501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:28.785501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:28.785501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:28.785501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:28.786392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:28.786392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:28.786392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:28.786392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:28.790843 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 429/840] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:28.791803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659136e-e1bf-44a8-a3d8-37e5a41cf99f None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:28.804812 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf2a71-41f0-4bee-b80a-c2c7acd81e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:28.806048 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf2a71-41f0-4bee-b80a-c2c7acd81e93 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:28.806506 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf2a71-41f0-4bee-b80a-c2c7acd81e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:28.806953 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf2a71-41f0-4bee-b80a-c2c7acd81e93 None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:28.807947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf2a71-41f0-4bee-b80a-c2c7acd81e93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:28.808441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf2a71-41f0-4bee-b80a-c2c7acd81e93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:28.810205 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf2a71-41f0-4bee-b80a-c2c7acd81e93 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:28.818640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:28.827784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 412/841] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:28.833914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf2a71-41f0-4bee-b80a-c2c7acd81e93 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:28.956595 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf2a71-41f0-4bee-b80a-c2c7acd81e93 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:29.007563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e3922-97c5-4220-9961-400764abceb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:29.008617 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e3922-97c5-4220-9961-400764abceb0 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:29.009160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e3922-97c5-4220-9961-400764abceb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:29.009654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e3922-97c5-4220-9961-400764abceb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:29.010736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e3922-97c5-4220-9961-400764abceb0 None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=24915625354f418eb7bc4d5ada85fdb0) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:29.011329 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e3922-97c5-4220-9961-400764abceb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:29.014013 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e3922-97c5-4220-9961-400764abceb0 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:29.027423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 413/842] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:53:28 2022] GET /identity/v3/role_assignments?user.id=24915625354f418eb7bc4d5ada85fdb0&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=053d1730ed764973abb1490894e07681 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:29.044128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659136e-e1bf-44a8-a3d8-37e5a41cf99f None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:29.084260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b21b6-9a29-453c-b437-70b53afcd2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:29.085087 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b21b6-9a29-453c-b437-70b53afcd2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:29.085497 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b21b6-9a29-453c-b437-70b53afcd2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:29.085993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b21b6-9a29-453c-b437-70b53afcd2c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:29.086896 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b21b6-9a29-453c-b437-70b53afcd2c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:29.087290 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b21b6-9a29-453c-b437-70b53afcd2c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:29.088830 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b21b6-9a29-453c-b437-70b53afcd2c1 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:29.104171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 430/843] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:29.112526 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e3922-97c5-4220-9961-400764abceb0 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:29.188730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:29.236515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e3922-97c5-4220-9961-400764abceb0 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:29.274108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343af8c1-3dba-4018-afbb-2f18b6faf2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:29.275246 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343af8c1-3dba-4018-afbb-2f18b6faf2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:29.275763 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343af8c1-3dba-4018-afbb-2f18b6faf2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:29.276232 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343af8c1-3dba-4018-afbb-2f18b6faf2c1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:29.284472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343af8c1-3dba-4018-afbb-2f18b6faf2c1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:29.286153 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343af8c1-3dba-4018-afbb-2f18b6faf2c1 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:29.301601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-343af8c1-3dba-4018-afbb-2f18b6faf2c1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 21:53:29.301601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:29.301601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:29.301601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:29.301601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:29.301601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:29.301601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:29.301601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:29.301601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:29.301601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:29.301601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:29.301601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:29.301601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:29.301601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:29.301601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:29.301601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:29.302757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:29.304601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:29.304601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:29.304601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:29.304601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:29.304601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:29.304601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:29.304601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:29.304601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:29.304601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 21:53:29.304601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:29.307016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 414/844] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:53:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:29.312747 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b21b6-9a29-453c-b437-70b53afcd2c1 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:29.349232 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:29.425509 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b21b6-9a29-453c-b437-70b53afcd2c1 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:29.461655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b538ba7-ec15-4bba-9dc3-960780b5d371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:29.462330 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b538ba7-ec15-4bba-9dc3-960780b5d371 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:29.462607 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b538ba7-ec15-4bba-9dc3-960780b5d371 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:29.462877 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b538ba7-ec15-4bba-9dc3-960780b5d371 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:29.463717 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b538ba7-ec15-4bba-9dc3-960780b5d371 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:29.464024 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b538ba7-ec15-4bba-9dc3-960780b5d371 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:29.465540 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b538ba7-ec15-4bba-9dc3-960780b5d371 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:29.485226 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 431/845] 10.222.0.44 () {56 vars in 1135 bytes} [Tue Jul 12 21:53:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:29.490882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343af8c1-3dba-4018-afbb-2f18b6faf2c1 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:29.599635 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343af8c1-3dba-4018-afbb-2f18b6faf2c1 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:29.625701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:29.640375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e6715-5653-4305-818f-e41534d59a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:29.641059 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e6715-5653-4305-818f-e41534d59a9e None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:29.641340 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e6715-5653-4305-818f-e41534d59a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:29.641615 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e6715-5653-4305-818f-e41534d59a9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:29.657946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9a562-12a2-4170-bfaa-eb5fd0074dcf None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:29.658251 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9a562-12a2-4170-bfaa-eb5fd0074dcf None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:29.658491 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9a562-12a2-4170-bfaa-eb5fd0074dcf None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:29.660882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 432/846] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:29.669461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376394d-3154-4cea-b9ec-cc5f57282858 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:29.669901 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376394d-3154-4cea-b9ec-cc5f57282858 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:29.670481 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376394d-3154-4cea-b9ec-cc5f57282858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:29.693465 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-b376394d-3154-4cea-b9ec-cc5f57282858 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:29.696387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b376394d-3154-4cea-b9ec-cc5f57282858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:29.719562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e6715-5653-4305-818f-e41534d59a9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:29.721283 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e6715-5653-4305-818f-e41534d59a9e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:29.779178 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 433/847] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:29.806476 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b376394d-3154-4cea-b9ec-cc5f57282858 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:29.822828 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:29.857861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 415/848] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:53:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:53:29.861077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e6715-5653-4305-818f-e41534d59a9e None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:29.896014 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:29.931699 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b376394d-3154-4cea-b9ec-cc5f57282858 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:29.975066 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630903a9-b48e-46fb-9ed4-fcaee83ed126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:29.976659 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630903a9-b48e-46fb-9ed4-fcaee83ed126 None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:29.976956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630903a9-b48e-46fb-9ed4-fcaee83ed126 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:29.977428 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630903a9-b48e-46fb-9ed4-fcaee83ed126 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:29.978269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630903a9-b48e-46fb-9ed4-fcaee83ed126 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:29.979761 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630903a9-b48e-46fb-9ed4-fcaee83ed126 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:30.003405 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 434/849] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Jul 12 21:53:29 2022] POST /identity/v3/services => generated 251 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 21:53:30.009702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e6715-5653-4305-818f-e41534d59a9e None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:30.055735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c585a9-d73a-405b-b1ad-13f29896e7f6 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:30.056289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c585a9-d73a-405b-b1ad-13f29896e7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:30.056289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c585a9-d73a-405b-b1ad-13f29896e7f6 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:30.058147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 435/850] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:30.063732 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad61fa-3c1c-46ae-b398-c8ef1c038d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:30.064548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad61fa-3c1c-46ae-b398-c8ef1c038d3a None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:30.064770 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c3d05-d8ca-4f55-8a07-c0d8c9b7c814 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:30.064898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad61fa-3c1c-46ae-b398-c8ef1c038d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:30.065066 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c3d05-d8ca-4f55-8a07-c0d8c9b7c814 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:30.065161 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad61fa-3c1c-46ae-b398-c8ef1c038d3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:30.065304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c3d05-d8ca-4f55-8a07-c0d8c9b7c814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:30.081050 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad61fa-3c1c-46ae-b398-c8ef1c038d3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:30.082611 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-953c3d05-d8ca-4f55-8a07-c0d8c9b7c814 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:30.083360 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad61fa-3c1c-46ae-b398-c8ef1c038d3a None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:30.083847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-953c3d05-d8ca-4f55-8a07-c0d8c9b7c814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:30.109304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-e5ad61fa-3c1c-46ae-b398-c8ef1c038d3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:30.109304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:30.109304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:30.109304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:30.109304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:30.109304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:30.109304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:30.109304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:30.109304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:30.109304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:30.109304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:30.109304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:30.109304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:30.109304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:30.109304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:30.110308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:30.110308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:30.110308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:30.110308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:30.110308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:30.110308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:30.110308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:30.110308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:30.110308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:30.110308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:30.110308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:30.110308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:30.110308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:30.110308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:30.110308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:30.110308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:30.110308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:30.111195 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:30.111195 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:30.111195 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:30.111195 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:30.114106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 416/851] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:30.120621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad61fa-3c1c-46ae-b398-c8ef1c038d3a None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:30.173073 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 436/852] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:30 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:53:30.187600 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c3d05-d8ca-4f55-8a07-c0d8c9b7c814 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:30.190801 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:30.212186 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:30.257749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad61fa-3c1c-46ae-b398-c8ef1c038d3a None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:30.305411 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0449229-da47-4e7e-8b7e-4756febc74aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:30.306100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0449229-da47-4e7e-8b7e-4756febc74aa None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:30.306382 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0449229-da47-4e7e-8b7e-4756febc74aa None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:30.306660 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0449229-da47-4e7e-8b7e-4756febc74aa None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:30.306858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c3d05-d8ca-4f55-8a07-c0d8c9b7c814 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:30.307408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0449229-da47-4e7e-8b7e-4756febc74aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:30.307713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0449229-da47-4e7e-8b7e-4756febc74aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:30.310514 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0449229-da47-4e7e-8b7e-4756febc74aa None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:30.326086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 417/853] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:30.331837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0449229-da47-4e7e-8b7e-4756febc74aa None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:30.351291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd21e0-1d05-41d1-8d66-c99c9a62afb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:30.352289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd21e0-1d05-41d1-8d66-c99c9a62afb0 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:30.352566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd21e0-1d05-41d1-8d66-c99c9a62afb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:30.352869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd21e0-1d05-41d1-8d66-c99c9a62afb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:30.364192 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd21e0-1d05-41d1-8d66-c99c9a62afb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:30.372798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-e9fd21e0-1d05-41d1-8d66-c99c9a62afb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:30.372798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:30.372798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:30.372798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:30.372798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:30.372798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:30.372798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:30.372798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:30.372798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:30.372798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:30.372798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:30.372798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:30.372798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:30.372798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:30.372798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:30.372798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:30.373822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:30.373822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:30.373822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:30.373822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:30.373822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:30.373822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:30.373822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:30.373822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:30.373822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:30.373822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:30.373822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:30.373822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:30.373822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:30.373822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:30.373822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:30.373822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:30.375080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:30.375080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:30.375080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:30.375080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:30.375080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:30.375080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:30.375080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:30.375080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:30.375080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:30.375080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:30.375080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:53:30.375080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:30.375080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 437/854] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:53:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:30.380177 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd21e0-1d05-41d1-8d66-c99c9a62afb0 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:30.469795 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0449229-da47-4e7e-8b7e-4756febc74aa None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:30.471578 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:30.512233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:30.519008 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc7503-a889-45ce-bcfa-2c8da8762172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:30.519892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc7503-a889-45ce-bcfa-2c8da8762172 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:30.520252 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc7503-a889-45ce-bcfa-2c8da8762172 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:30.520602 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc7503-a889-45ce-bcfa-2c8da8762172 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:30.522478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd21e0-1d05-41d1-8d66-c99c9a62afb0 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:30.527942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc7503-a889-45ce-bcfa-2c8da8762172 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:30.529450 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc7503-a889-45ce-bcfa-2c8da8762172 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:30.544757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-43dc7503-a889-45ce-bcfa-2c8da8762172 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:30.544757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:30.544757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:30.544757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:30.544757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:30.544757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:30.544757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:30.544757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:30.544757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:30.544757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:30.544757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:30.544757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:30.544757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:30.544757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:30.544757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:30.546872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:30.546872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:30.546872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:30.546872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:30.546872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:30.546872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:30.546872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:30.546872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:30.546872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:30.546872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:30.546872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:30.546872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:30.546872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:30.546872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:30.546872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:30.546872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:30.546872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:30.549084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:30.549084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:30.549084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:30.549084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:30.549084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:30.549084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:30.549084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:30.549084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:30.552570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 418/855] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:30.552570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc7503-a889-45ce-bcfa-2c8da8762172 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:30.561703 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ffaf45-1f04-41c3-a520-3b3ef6ba6c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:30.562441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffaf45-1f04-41c3-a520-3b3ef6ba6c78 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:30.562725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffaf45-1f04-41c3-a520-3b3ef6ba6c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:30.563111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffaf45-1f04-41c3-a520-3b3ef6ba6c78 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:30.563925 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ffaf45-1f04-41c3-a520-3b3ef6ba6c78 None admin] RBAC: Adding query filter params (name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:30.564242 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ffaf45-1f04-41c3-a520-3b3ef6ba6c78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:30.565756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ffaf45-1f04-41c3-a520-3b3ef6ba6c78 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:30.580381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 438/856] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:53:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:30.581428 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ffaf45-1f04-41c3-a520-3b3ef6ba6c78 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:30.658437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc7503-a889-45ce-bcfa-2c8da8762172 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:30.693175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ffaf45-1f04-41c3-a520-3b3ef6ba6c78 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:30.695171 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d21301-cc85-45e2-801f-4e31a1295137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:30.695879 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d21301-cc85-45e2-801f-4e31a1295137 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:30.696152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d21301-cc85-45e2-801f-4e31a1295137 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:30.696436 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d21301-cc85-45e2-801f-4e31a1295137 None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:30.697265 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d21301-cc85-45e2-801f-4e31a1295137 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:30.697579 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d21301-cc85-45e2-801f-4e31a1295137 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:30.699033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d21301-cc85-45e2-801f-4e31a1295137 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:30.717092 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 419/857] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:30.719884 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d21301-cc85-45e2-801f-4e31a1295137 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:30.724935 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:30.739088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64271aa-6575-48d1-874e-57dab166e7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:30.741108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64271aa-6575-48d1-874e-57dab166e7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:30.742553 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64271aa-6575-48d1-874e-57dab166e7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:30.743318 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64271aa-6575-48d1-874e-57dab166e7d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:30.756054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64271aa-6575-48d1-874e-57dab166e7d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:30.757951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64271aa-6575-48d1-874e-57dab166e7d6 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:30.777385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-e64271aa-6575-48d1-874e-57dab166e7d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:30.777385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:30.777385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:30.777385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:30.777385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:30.777385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:30.777385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:30.777385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:30.777385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:30.777385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:30.777385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:30.777385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:30.777385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:30.777385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:30.777385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:30.778505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:30.778505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:30.778505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:30.778505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:30.778505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:30.778505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:30.778505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:30.778505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:30.778505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:30.778505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:30.778505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:30.778505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:30.778505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:30.778505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:30.778505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:30.778505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:30.778505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:30.779527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:30.779527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:30.779527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:30.779527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:30.785873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 439/858] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:30.786980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:30.787735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64271aa-6575-48d1-874e-57dab166e7d6 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:30.851554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d21301-cc85-45e2-801f-4e31a1295137 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:30.889170 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7572f-0030-482f-a4eb-e95bf51bc43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:30.889842 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7572f-0030-482f-a4eb-e95bf51bc43f None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:30.890214 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7572f-0030-482f-a4eb-e95bf51bc43f None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:30.890481 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7572f-0030-482f-a4eb-e95bf51bc43f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:30.891330 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7572f-0030-482f-a4eb-e95bf51bc43f None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=837bb5ac18554dfdbbcc776689639494) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:30.891645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7572f-0030-482f-a4eb-e95bf51bc43f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:30.893120 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7572f-0030-482f-a4eb-e95bf51bc43f None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:30.906577 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64271aa-6575-48d1-874e-57dab166e7d6 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:30.908328 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 420/859] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:53:30 2022] GET /identity/v3/role_assignments?user.id=837bb5ac18554dfdbbcc776689639494&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=053d1730ed764973abb1490894e07681 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:30.951534 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbacb30e-47d8-4f05-b452-393486c8ee59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:30.952522 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbacb30e-47d8-4f05-b452-393486c8ee59 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:30.953009 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbacb30e-47d8-4f05-b452-393486c8ee59 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:30.953477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbacb30e-47d8-4f05-b452-393486c8ee59 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:30.954391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbacb30e-47d8-4f05-b452-393486c8ee59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:30.954895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbacb30e-47d8-4f05-b452-393486c8ee59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:30.956490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbacb30e-47d8-4f05-b452-393486c8ee59 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:30.972980 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 440/860] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:30.978665 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7572f-0030-482f-a4eb-e95bf51bc43f None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:31.110886 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7572f-0030-482f-a4eb-e95bf51bc43f None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:31.133535 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:31.159880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16b404-0813-4ff5-95f7-6653d1de511e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:31.160721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16b404-0813-4ff5-95f7-6653d1de511e None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:31.161021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16b404-0813-4ff5-95f7-6653d1de511e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:31.161305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16b404-0813-4ff5-95f7-6653d1de511e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:31.169692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc16b404-0813-4ff5-95f7-6653d1de511e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:31.171154 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc16b404-0813-4ff5-95f7-6653d1de511e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:31.186462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-dc16b404-0813-4ff5-95f7-6653d1de511e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 21:53:31.186462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:31.186462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:31.186462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:31.186462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:31.186462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:31.186462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:31.186462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:31.186462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:31.186462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:31.186462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:31.186462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:31.186462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:31.186462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:31.186462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:31.186462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:31.187464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:31.188414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:31.188414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:31.188414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:31.188414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:31.188414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:31.188414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:31.188414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:31.188414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:31.188414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 21:53:31.188414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:31.188414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 421/861] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Jul 12 21:53:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:31.196174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbacb30e-47d8-4f05-b452-393486c8ee59 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:31.242029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:31.281473 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56978c6-326b-486c-8ba5-326e02bcf565 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:31.282007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56978c6-326b-486c-8ba5-326e02bcf565 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:31.282007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56978c6-326b-486c-8ba5-326e02bcf565 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:31.283695 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 422/862] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:53:31.290671 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84b296-5775-4f42-a026-496155ece4c4 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:31.291092 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84b296-5775-4f42-a026-496155ece4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:31.291799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84b296-5775-4f42-a026-496155ece4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:31.308899 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-0a84b296-5775-4f42-a026-496155ece4c4 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:31.310261 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0a84b296-5775-4f42-a026-496155ece4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:31.311011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbacb30e-47d8-4f05-b452-393486c8ee59 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:31.347638 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf1277-56d0-4117-9d3d-ff74cf186be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:31.348887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf1277-56d0-4117-9d3d-ff74cf186be0 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:31.349362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf1277-56d0-4117-9d3d-ff74cf186be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:31.349791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf1277-56d0-4117-9d3d-ff74cf186be0 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:31.351863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf1277-56d0-4117-9d3d-ff74cf186be0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:31.352401 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf1277-56d0-4117-9d3d-ff74cf186be0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:31.354295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf1277-56d0-4117-9d3d-ff74cf186be0 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:31.374301 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 441/863] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Jul 12 21:53:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:31.383169 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf1277-56d0-4117-9d3d-ff74cf186be0 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:31.391730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 423/864] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:31 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:31.408942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84b296-5775-4f42-a026-496155ece4c4 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:31.489825 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:31.498438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf1277-56d0-4117-9d3d-ff74cf186be0 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:31.531533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84b296-5775-4f42-a026-496155ece4c4 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:31.534297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:31.545037 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c75780-471c-4241-91ef-b2485d533d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:31.545987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c75780-471c-4241-91ef-b2485d533d1a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:31.546272 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c75780-471c-4241-91ef-b2485d533d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:31.546548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c75780-471c-4241-91ef-b2485d533d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:31.570048 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bf8ad-f86d-4000-a245-873501b7f5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:31.571075 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bf8ad-f86d-4000-a245-873501b7f5ec None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:31.571457 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bf8ad-f86d-4000-a245-873501b7f5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:31.571838 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bf8ad-f86d-4000-a245-873501b7f5ec None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:31.572735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bf8ad-f86d-4000-a245-873501b7f5ec None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:31.574270 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bf8ad-f86d-4000-a245-873501b7f5ec None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:31.588368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-c29bf8ad-f86d-4000-a245-873501b7f5ec None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 21:53:31.588368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:31.588368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:31.588368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:31.588368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:31.588368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:31.588368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:31.588368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:31.588368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 21:53:31.588368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:31.588368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:31.588368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:31.588368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:31.588368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:31.588368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:31.588368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:31.589318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:31.589318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:31.589318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:31.589318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:31.589318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:31.589318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:31.589318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:31.589318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:31.589318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:31.589318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:31.589318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:31.589318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:31.589318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:31.589318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:31.589318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:31.589318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:31.590148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:31.590148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:31.590148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:31.590148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:31.590148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:31.590148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:31.590148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:31.590148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:31.590148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:31.590148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:31.590148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:31.590148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:31.590148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:31.590148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 21:53:31.590148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:31.594914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 424/865] 10.222.0.44 () {56 vars in 1132 bytes} [Tue Jul 12 21:53:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:31.596199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bf8ad-f86d-4000-a245-873501b7f5ec None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:31.626363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c75780-471c-4241-91ef-b2485d533d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:31.628375 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c75780-471c-4241-91ef-b2485d533d1a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:31.711894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bf8ad-f86d-4000-a245-873501b7f5ec None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:31.736205 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:31.746356 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 442/866] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:53:31 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:53:31.748200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec0663-13db-49e9-a2c3-35799123a970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:31.757402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec0663-13db-49e9-a2c3-35799123a970 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:31.757402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec0663-13db-49e9-a2c3-35799123a970 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:31.757402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec0663-13db-49e9-a2c3-35799123a970 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:31.757402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec0663-13db-49e9-a2c3-35799123a970 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:31.757402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec0663-13db-49e9-a2c3-35799123a970 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:31.757888 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c75780-471c-4241-91ef-b2485d533d1a None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:31.757964 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec0663-13db-49e9-a2c3-35799123a970 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:31.758137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 425/867] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Jul 12 21:53:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:31.764421 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec0663-13db-49e9-a2c3-35799123a970 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:31.803118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:31.870622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c75780-471c-4241-91ef-b2485d533d1a None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:31.877043 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec0663-13db-49e9-a2c3-35799123a970 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:31.907785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e95f4c-2108-4ea9-ba17-48ec869a62ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:31.908574 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e95f4c-2108-4ea9-ba17-48ec869a62ea None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:31.908925 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e95f4c-2108-4ea9-ba17-48ec869a62ea None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:31.909207 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e95f4c-2108-4ea9-ba17-48ec869a62ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:31.914252 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097935f7-a125-41e3-b4f2-1f7fe41abd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:31.915085 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097935f7-a125-41e3-b4f2-1f7fe41abd81 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:31.915511 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097935f7-a125-41e3-b4f2-1f7fe41abd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:31.915947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097935f7-a125-41e3-b4f2-1f7fe41abd81 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:31.916823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097935f7-a125-41e3-b4f2-1f7fe41abd81 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:31.917272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097935f7-a125-41e3-b4f2-1f7fe41abd81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:31.918841 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097935f7-a125-41e3-b4f2-1f7fe41abd81 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:31.921372 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e95f4c-2108-4ea9-ba17-48ec869a62ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:31.922814 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e95f4c-2108-4ea9-ba17-48ec869a62ea None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:31.924458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 426/868] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Jul 12 21:53:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:31.942854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-a2e95f4c-2108-4ea9-ba17-48ec869a62ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:31.942854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:31.942854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:31.942854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:31.942854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:31.942854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:31.942854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:31.942854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:31.942854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:31.942854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:31.942854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:31.942854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:31.942854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:31.942854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:31.942854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:31.943963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:31.943963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:31.943963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:31.943963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:31.943963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:31.943963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:31.943963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:31.943963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:31.943963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:31.943963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:31.943963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:31.943963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:31.943963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:31.943963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:31.943963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:31.943963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:31.943963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:31.945549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:31.945549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:31.945549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:31.945549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:31.945549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 443/869] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:31.951024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097935f7-a125-41e3-b4f2-1f7fe41abd81 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:32.076597 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:32.083961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097935f7-a125-41e3-b4f2-1f7fe41abd81 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:32.123537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29b25c-cfa4-4efa-aee0-7515dcc296ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:32.124622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b25c-cfa4-4efa-aee0-7515dcc296ec None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:32.125143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b25c-cfa4-4efa-aee0-7515dcc296ec None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:32.125653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b25c-cfa4-4efa-aee0-7515dcc296ec None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:32.126617 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29b25c-cfa4-4efa-aee0-7515dcc296ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:32.127140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29b25c-cfa4-4efa-aee0-7515dcc296ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:32.128908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29b25c-cfa4-4efa-aee0-7515dcc296ec None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:32.144115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 427/870] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:32.151716 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e95f4c-2108-4ea9-ba17-48ec869a62ea None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:32.203708 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:32.208685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c96001-e949-441c-a196-a6e806412987 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:32.209111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c96001-e949-441c-a196-a6e806412987 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:32.209486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c96001-e949-441c-a196-a6e806412987 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:32.211243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 428/871] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:32.216868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512b1fe-d363-46a4-b160-ffe68a4cd1ce None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:32.217250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512b1fe-d363-46a4-b160-ffe68a4cd1ce None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:32.217626 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512b1fe-d363-46a4-b160-ffe68a4cd1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:32.236031 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-5512b1fe-d363-46a4-b160-ffe68a4cd1ce None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:32.237457 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-5512b1fe-d363-46a4-b160-ffe68a4cd1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:32.259475 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e95f4c-2108-4ea9-ba17-48ec869a62ea None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:32.297857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfd035-3203-4162-911f-3a14ef083b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:32.298661 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfd035-3203-4162-911f-3a14ef083b7d None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:32.298950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfd035-3203-4162-911f-3a14ef083b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:32.299231 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfd035-3203-4162-911f-3a14ef083b7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:32.306611 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dfd035-3203-4162-911f-3a14ef083b7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:32.308098 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dfd035-3203-4162-911f-3a14ef083b7d None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:32.313737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 429/872] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:32 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:32.323307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-e7dfd035-3203-4162-911f-3a14ef083b7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:32.323307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:32.323307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:32.323307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:32.323307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:32.323307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:32.323307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:32.323307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:32.323307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:32.323307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:32.323307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:32.323307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:32.323307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:32.323307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:32.323307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:32.324323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:32.324323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:32.324323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:32.324323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:32.324323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:32.324323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:32.324323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:32.324323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:32.324323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:32.324323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:32.324323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:32.324323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:32.324323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:32.324323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:32.324323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:32.324323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:32.324323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:32.325346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:32.325346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:32.325346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:32.325346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:32.325346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:32.325346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:32.325346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:32.325346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:32.329255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 444/873] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:32.330708 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512b1fe-d363-46a4-b160-ffe68a4cd1ce None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:32.340698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfd035-3203-4162-911f-3a14ef083b7d None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:32.422818 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:32.441814 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512b1fe-d363-46a4-b160-ffe68a4cd1ce None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:32.451604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfd035-3203-4162-911f-3a14ef083b7d None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:32.461353 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:32.488129 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c59204-d89d-4b03-9596-2c209c5b879a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:32.489083 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c59204-d89d-4b03-9596-2c209c5b879a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:32.489376 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c59204-d89d-4b03-9596-2c209c5b879a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:32.489654 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c59204-d89d-4b03-9596-2c209c5b879a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:32.490312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ced0c5-1b3c-4ad2-8c59-68bd89b1edbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:32.490463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c59204-d89d-4b03-9596-2c209c5b879a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:32.490853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ced0c5-1b3c-4ad2-8c59-68bd89b1edbb None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:32.491787 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ced0c5-1b3c-4ad2-8c59-68bd89b1edbb None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:32.492157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ced0c5-1b3c-4ad2-8c59-68bd89b1edbb None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:32.492628 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c59204-d89d-4b03-9596-2c209c5b879a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:32.493168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ced0c5-1b3c-4ad2-8c59-68bd89b1edbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:32.493570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ced0c5-1b3c-4ad2-8c59-68bd89b1edbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:32.495316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ced0c5-1b3c-4ad2-8c59-68bd89b1edbb None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:32.505698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-d8c59204-d89d-4b03-9596-2c209c5b879a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 21:53:32.505698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:32.505698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:32.505698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:32.505698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:32.505698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:32.505698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:32.505698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:32.505698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 21:53:32.505698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:32.505698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:32.505698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:32.505698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:32.505698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:32.505698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:32.505698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:32.505698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:32.507001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:32.507001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:32.507001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:32.507001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:32.507001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:32.507001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:32.507001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:32.507001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:32.507001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:32.507001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:32.507001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:32.507001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:32.507001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:32.507001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:32.507001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:32.507001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:32.507939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:32.507939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:32.507939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:32.507939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:32.507939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:32.507939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:32.507939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:32.507939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:32.507939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:32.507939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:32.507939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:32.507939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:32.507939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 21:53:32.507939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:32.507939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 445/874] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:53:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:32.514711 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 430/875] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:32.515523 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c59204-d89d-4b03-9596-2c209c5b879a None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:32.515627 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ced0c5-1b3c-4ad2-8c59-68bd89b1edbb None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:32.619205 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ced0c5-1b3c-4ad2-8c59-68bd89b1edbb None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:32.624409 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c59204-d89d-4b03-9596-2c209c5b879a None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:32.653933 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467e824-fa4a-4d52-9d21-c217894d80f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:32.654630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467e824-fa4a-4d52-9d21-c217894d80f8 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:32.655033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467e824-fa4a-4d52-9d21-c217894d80f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:32.655152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467e824-fa4a-4d52-9d21-c217894d80f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:32.656012 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467e824-fa4a-4d52-9d21-c217894d80f8 None admin] RBAC: Adding query filter params (role.id=f1e42ea5a1c440bba915b459d2744858, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=8d6b823cc7dc434986f77739fa0259ea) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:32.656364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467e824-fa4a-4d52-9d21-c217894d80f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:32.657752 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467e824-fa4a-4d52-9d21-c217894d80f8 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:32.664350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7f386-fbe2-4043-81bf-ed56910ee61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:32.665316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7f386-fbe2-4043-81bf-ed56910ee61b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:32.665736 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7f386-fbe2-4043-81bf-ed56910ee61b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:32.666161 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7f386-fbe2-4043-81bf-ed56910ee61b None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:32.667038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7f386-fbe2-4043-81bf-ed56910ee61b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:32.667493 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7f386-fbe2-4043-81bf-ed56910ee61b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:32.669158 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7f386-fbe2-4043-81bf-ed56910ee61b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:32.673201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 431/876] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:53:32 2022] GET /identity/v3/role_assignments?user.id=8d6b823cc7dc434986f77739fa0259ea&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=f1e42ea5a1c440bba915b459d2744858 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:32.679954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 446/877] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:53:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:32.681735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467e824-fa4a-4d52-9d21-c217894d80f8 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:32.697163 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:32.811382 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467e824-fa4a-4d52-9d21-c217894d80f8 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:32.848736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a435b-1097-4e13-a6f6-a3644fa17283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:32.849408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a435b-1097-4e13-a6f6-a3644fa17283 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:32.849679 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a435b-1097-4e13-a6f6-a3644fa17283 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:32.849953 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a435b-1097-4e13-a6f6-a3644fa17283 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:32.851189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a435b-1097-4e13-a6f6-a3644fa17283 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:32.851493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a435b-1097-4e13-a6f6-a3644fa17283 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:32.852910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a435b-1097-4e13-a6f6-a3644fa17283 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:32.858652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 432/878] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:53:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:32.864342 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7f386-fbe2-4043-81bf-ed56910ee61b None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:32.908855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:32.983647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7f386-fbe2-4043-81bf-ed56910ee61b None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:33.020520 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e061dcf-8ef5-47fe-aa3a-2d9f560840c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:33.021417 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e061dcf-8ef5-47fe-aa3a-2d9f560840c4 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:33.021738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e061dcf-8ef5-47fe-aa3a-2d9f560840c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:33.022047 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e061dcf-8ef5-47fe-aa3a-2d9f560840c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:33.023565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e061dcf-8ef5-47fe-aa3a-2d9f560840c4 None admin] RBAC: Adding query filter params (interface=public, service_id=4bf4291589124e92ac6a9719cfe9ea28, region_id=RegionOne) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:33.024061 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e061dcf-8ef5-47fe-aa3a-2d9f560840c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:33.025672 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e061dcf-8ef5-47fe-aa3a-2d9f560840c4 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:33.032319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 447/879] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Jul 12 21:53:32 2022] GET /identity/v3/endpoints?service_id=4bf4291589124e92ac6a9719cfe9ea28&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:33.039810 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a435b-1097-4e13-a6f6-a3644fa17283 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:33.132934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:33.136872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87635ed-27fb-42be-b4f6-890a4b72e212 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:33.137186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87635ed-27fb-42be-b4f6-890a4b72e212 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:33.137446 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87635ed-27fb-42be-b4f6-890a4b72e212 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:33.139188 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 448/880] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:33.144433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b3224-5305-4274-886d-e40dc4a41300 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:33.144742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b3224-5305-4274-886d-e40dc4a41300 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:33.145042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b3224-5305-4274-886d-e40dc4a41300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:33.159444 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a435b-1097-4e13-a6f6-a3644fa17283 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:33.163597 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-af9b3224-5305-4274-886d-e40dc4a41300 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:33.164838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-af9b3224-5305-4274-886d-e40dc4a41300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:33.202407 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e9bca-3b30-4259-b48f-fe9ccefef704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:33.203324 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e9bca-3b30-4259-b48f-fe9ccefef704 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:33.203625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e9bca-3b30-4259-b48f-fe9ccefef704 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:33.203902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e9bca-3b30-4259-b48f-fe9ccefef704 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:33.205124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e9bca-3b30-4259-b48f-fe9ccefef704 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:33.206853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e9bca-3b30-4259-b48f-fe9ccefef704 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:33.214730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 433/881] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Jul 12 21:53:33 2022] GET /identity/v3/services => generated 1011 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 21:53:33.251123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 449/882] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:53:33.274857 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e9bca-3b30-4259-b48f-fe9ccefef704 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:33.386267 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e9bca-3b30-4259-b48f-fe9ccefef704 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:33.427491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a1720-82ab-4574-ac8f-7e5feb728ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:33.428987 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a1720-82ab-4574-ac8f-7e5feb728ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:33.429304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a1720-82ab-4574-ac8f-7e5feb728ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:33.429628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a1720-82ab-4574-ac8f-7e5feb728ab5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:33.430950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a1720-82ab-4574-ac8f-7e5feb728ab5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:33.432970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a1720-82ab-4574-ac8f-7e5feb728ab5 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:33.448094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-493a1720-82ab-4574-ac8f-7e5feb728ab5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 21:53:33.448094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:33.448094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:33.448094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:33.448094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:33.448094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:33.448094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:33.448094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:33.448094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 21:53:33.448094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:33.448094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:33.448094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:33.448094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:33.448094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:33.448094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:33.448094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:33.448094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:33.449142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:33.449142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:33.449142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:33.449142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:33.449142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:33.449142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:33.449142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:33.449142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:33.449142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:33.449142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:33.449142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:33.449142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:33.449142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:33.449142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:33.449142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:33.449142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:33.450224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:33.450224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:33.450224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:33.450224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:33.450224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:33.450224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:33.450224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:33.450224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:33.450224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:33.450224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:33.450224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:33.450224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:33.450224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 21:53:33.450224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:33.450224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 434/883] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Jul 12 21:53:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:33.455202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b3224-5305-4274-886d-e40dc4a41300 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:33.489821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:33.563580 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b3224-5305-4274-886d-e40dc4a41300 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:33.599650 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094172c1-c259-41bb-bbc4-565282bca810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:33.600323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094172c1-c259-41bb-bbc4-565282bca810 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:33.601240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094172c1-c259-41bb-bbc4-565282bca810 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:33.601598 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094172c1-c259-41bb-bbc4-565282bca810 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:33.602231 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094172c1-c259-41bb-bbc4-565282bca810 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:33.602539 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094172c1-c259-41bb-bbc4-565282bca810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:33.603919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094172c1-c259-41bb-bbc4-565282bca810 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:33.609447 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 450/884] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Jul 12 21:53:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:33.615054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a1720-82ab-4574-ac8f-7e5feb728ab5 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:33.617991 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:33.723503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a1720-82ab-4574-ac8f-7e5feb728ab5 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:33.758324 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdda8f3-7fff-4237-bc8c-9cc473f90e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:33.758990 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdda8f3-7fff-4237-bc8c-9cc473f90e73 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:33.759274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdda8f3-7fff-4237-bc8c-9cc473f90e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:33.759545 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdda8f3-7fff-4237-bc8c-9cc473f90e73 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:33.760250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdda8f3-7fff-4237-bc8c-9cc473f90e73 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:33.760555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdda8f3-7fff-4237-bc8c-9cc473f90e73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:33.761957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdda8f3-7fff-4237-bc8c-9cc473f90e73 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:33.767084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 435/885] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Jul 12 21:53:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:34.213168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb3c9c-a6ab-4141-b711-c5745fd804fd None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:34.213168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb3c9c-a6ab-4141-b711-c5745fd804fd None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:34.213168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb3c9c-a6ab-4141-b711-c5745fd804fd None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:34.213168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 451/886] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:34.221532 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb873e-9f8c-459c-97f6-aa6d77772b9c None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:34.221532 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb873e-9f8c-459c-97f6-aa6d77772b9c None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:34.221532 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb873e-9f8c-459c-97f6-aa6d77772b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:34.244738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-0fbb873e-9f8c-459c-97f6-aa6d77772b9c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:34.244738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0fbb873e-9f8c-459c-97f6-aa6d77772b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:34.304306 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:34.314094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 436/887] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:34.338121 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb3c9c-a6ab-4141-b711-c5745fd804fd None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:34.443205 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:34.452308 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb3c9c-a6ab-4141-b711-c5745fd804fd None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:34.490191 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d04907-b7d7-48bc-a790-2349033af582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:34.490823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d04907-b7d7-48bc-a790-2349033af582 None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:34.491665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d04907-b7d7-48bc-a790-2349033af582 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:34.492034 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d04907-b7d7-48bc-a790-2349033af582 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:34.492805 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d04907-b7d7-48bc-a790-2349033af582 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:34.494255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d04907-b7d7-48bc-a790-2349033af582 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:34.510124 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 452/888] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Jul 12 21:53:34 2022] POST /identity/v3/services => generated 269 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:53:34.968983 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ca092-2768-4a41-add5-3fac23b37687 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:34.969443 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ca092-2768-4a41-add5-3fac23b37687 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:34.969532 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ca092-2768-4a41-add5-3fac23b37687 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:34.971167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 437/889] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:34.976931 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c2aeb-cf4e-4e69-b5cb-f9abf4982959 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:34.977226 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c2aeb-cf4e-4e69-b5cb-f9abf4982959 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:34.977507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c2aeb-cf4e-4e69-b5cb-f9abf4982959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:34.995316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-280c2aeb-cf4e-4e69-b5cb-f9abf4982959 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:34.996519 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-280c2aeb-cf4e-4e69-b5cb-f9abf4982959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:35.075152 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 453/890] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:35.097966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ca092-2768-4a41-add5-3fac23b37687 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:35.212508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ca092-2768-4a41-add5-3fac23b37687 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:35.249099 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c97fb-c129-4cfa-9caf-53a2521281c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:35.250280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c97fb-c129-4cfa-9caf-53a2521281c7 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:35.250768 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c97fb-c129-4cfa-9caf-53a2521281c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:35.251250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c97fb-c129-4cfa-9caf-53a2521281c7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:35.262709 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c97fb-c129-4cfa-9caf-53a2521281c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:35.271338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-f77c97fb-c129-4cfa-9caf-53a2521281c7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 21:53:35.271338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:35.271338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:35.271338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:35.271338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:35.271338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:35.271338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:35.271338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:35.271338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:35.271338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:35.271338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:35.271338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:35.271338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:35.271338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:35.271338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:35.271338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:35.272866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:35.272866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:35.272866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:35.272866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:35.272866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:35.272866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:35.272866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:35.272866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:35.272866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:35.272866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:35.272866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:35.272866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:35.272866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:35.272866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:35.272866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:35.272866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:35.274073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:35.274073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:35.274073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:35.274073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:35.274073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:35.274073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:35.274073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:35.274073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:35.274073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:35.274073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:35.274073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 21:53:35.274073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:35.276529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 438/891] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Jul 12 21:53:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:35.284762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c2aeb-cf4e-4e69-b5cb-f9abf4982959 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:35.372073 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:35.385759 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09ab26-124c-40dd-b777-e65b5e57768c None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:35.386387 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09ab26-124c-40dd-b777-e65b5e57768c None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:35.387182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09ab26-124c-40dd-b777-e65b5e57768c None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:35.389157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 439/892] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:35.394295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c2aeb-cf4e-4e69-b5cb-f9abf4982959 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:35.394450 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ca538-20c7-449e-9667-bbecf1fef7fa None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:35.394825 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ca538-20c7-449e-9667-bbecf1fef7fa None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:35.395210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ca538-20c7-449e-9667-bbecf1fef7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:35.413462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-db5ca538-20c7-449e-9667-bbecf1fef7fa None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:35.414821 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-db5ca538-20c7-449e-9667-bbecf1fef7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:35.433375 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dd5da-49fd-4de9-ab3d-3699c2d7925c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:35.434268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd5da-49fd-4de9-ab3d-3699c2d7925c None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:35.434753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd5da-49fd-4de9-ab3d-3699c2d7925c None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:35.435185 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd5da-49fd-4de9-ab3d-3699c2d7925c None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:35.436103 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dd5da-49fd-4de9-ab3d-3699c2d7925c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:35.436609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dd5da-49fd-4de9-ab3d-3699c2d7925c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:35.438195 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dd5da-49fd-4de9-ab3d-3699c2d7925c None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:35.448096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 454/893] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Jul 12 21:53:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:35.453291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dd5da-49fd-4de9-ab3d-3699c2d7925c None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:35.468795 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:35.493008 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 440/894] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:53:35.511168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ca538-20c7-449e-9667-bbecf1fef7fa None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:35.565297 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dd5da-49fd-4de9-ab3d-3699c2d7925c None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:35.603632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817b996-984f-476e-ae04-0139057735a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:35.604663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817b996-984f-476e-ae04-0139057735a2 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:35.605113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817b996-984f-476e-ae04-0139057735a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:35.605525 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817b996-984f-476e-ae04-0139057735a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:35.617078 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ca538-20c7-449e-9667-bbecf1fef7fa None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:35.618261 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817b996-984f-476e-ae04-0139057735a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:35.620100 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817b996-984f-476e-ae04-0139057735a2 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:35.622162 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:35.642039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-2817b996-984f-476e-ae04-0139057735a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:35.642039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:35.642039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:35.642039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:35.642039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:35.642039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:35.642039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:35.642039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:35.642039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:35.642039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:35.642039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:35.642039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:35.642039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:35.642039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:35.642039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:35.642975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:35.642975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:35.642975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:35.642975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:35.642975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:35.642975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:35.642975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:35.642975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:35.642975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:35.642975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:35.642975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:35.642975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:35.642975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:35.642975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:35.642975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:35.642975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:35.642975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:35.643836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:35.643836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:35.643836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:35.643836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:35.646400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 455/895] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:35.651934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817b996-984f-476e-ae04-0139057735a2 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:35.655695 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ecdca-4e10-4471-a7a5-c480f56c0c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:35.656604 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ecdca-4e10-4471-a7a5-c480f56c0c73 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:35.657658 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ecdca-4e10-4471-a7a5-c480f56c0c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:35.657658 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ecdca-4e10-4471-a7a5-c480f56c0c73 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:35.657979 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ecdca-4e10-4471-a7a5-c480f56c0c73 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:35.659500 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ecdca-4e10-4471-a7a5-c480f56c0c73 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:35.673167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-a36ecdca-4e10-4471-a7a5-c480f56c0c73 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 21:53:35.673167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:35.673167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:35.673167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:35.673167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:35.673167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:35.673167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:35.673167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:35.673167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 21:53:35.673167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:35.673167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:35.673167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:35.673167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:35.673167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:35.673167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:35.673167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:35.673167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:35.674201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:35.674201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:35.674201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:35.674201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:35.674201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:35.674201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:35.674201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:35.674201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:35.674201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:35.674201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:35.674201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:35.674201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:35.674201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:35.674201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:35.674201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:35.674201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:35.675032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:35.675032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:35.675032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:35.675032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:35.675032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:35.675032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:35.675032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:35.675032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:35.675032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:35.675032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:35.675032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:35.675032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:35.675032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 21:53:35.675032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:35.680099 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 441/896] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:53:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:35.681249 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ecdca-4e10-4471-a7a5-c480f56c0c73 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:35.738749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:35.795218 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ecdca-4e10-4471-a7a5-c480f56c0c73 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:35.799209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817b996-984f-476e-ae04-0139057735a2 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:35.833453 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f9e68-446c-4519-9d40-446b70a39b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:35.834312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f9e68-446c-4519-9d40-446b70a39b2c None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:35.834730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f9e68-446c-4519-9d40-446b70a39b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:35.835075 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f9e68-446c-4519-9d40-446b70a39b2c None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:35.835863 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f9e68-446c-4519-9d40-446b70a39b2c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:35.836281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f9e68-446c-4519-9d40-446b70a39b2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:35.837756 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f9e68-446c-4519-9d40-446b70a39b2c None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:35.846316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca90db2-fb06-4733-a860-62a24e49ac68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:35.847247 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca90db2-fb06-4733-a860-62a24e49ac68 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:35.847577 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 442/897] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:53:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:35.847897 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca90db2-fb06-4733-a860-62a24e49ac68 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:35.848348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca90db2-fb06-4733-a860-62a24e49ac68 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:35.848647 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f9e68-446c-4519-9d40-446b70a39b2c None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:35.849587 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca90db2-fb06-4733-a860-62a24e49ac68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:35.850075 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca90db2-fb06-4733-a860-62a24e49ac68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:35.852698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca90db2-fb06-4733-a860-62a24e49ac68 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:35.870437 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 456/898] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:35.876750 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca90db2-fb06-4733-a860-62a24e49ac68 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:35.938009 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:35.962696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f9e68-446c-4519-9d40-446b70a39b2c None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:35.978093 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:36.000414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e4b87-58e5-4835-b66a-1bd60026ceed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:36.001103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e4b87-58e5-4835-b66a-1bd60026ceed None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:36.001393 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e4b87-58e5-4835-b66a-1bd60026ceed None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:36.001673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e4b87-58e5-4835-b66a-1bd60026ceed None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:36.002383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e4b87-58e5-4835-b66a-1bd60026ceed None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:36.002695 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e4b87-58e5-4835-b66a-1bd60026ceed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:36.004072 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e4b87-58e5-4835-b66a-1bd60026ceed None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:36.005088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca90db2-fb06-4733-a860-62a24e49ac68 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:36.009704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 443/899] 10.222.0.44 () {56 vars in 1114 bytes} [Tue Jul 12 21:53:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:36.016027 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44307c5-2d51-45b3-9ec0-4525c99bade8 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:36.016325 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44307c5-2d51-45b3-9ec0-4525c99bade8 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:36.016840 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44307c5-2d51-45b3-9ec0-4525c99bade8 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:36.020905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 444/900] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:36.020905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44307c5-2d51-45b3-9ec0-4525c99bade8 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:36.049258 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3631ff-8cd2-456e-8f20-f919f8e8d107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:36.049987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3631ff-8cd2-456e-8f20-f919f8e8d107 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:36.050273 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3631ff-8cd2-456e-8f20-f919f8e8d107 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:36.050557 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3631ff-8cd2-456e-8f20-f919f8e8d107 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:36.058012 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3631ff-8cd2-456e-8f20-f919f8e8d107 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:36.059464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3631ff-8cd2-456e-8f20-f919f8e8d107 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:36.074960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-ee3631ff-8cd2-456e-8f20-f919f8e8d107 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 21:53:36.074960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:36.074960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:36.074960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:36.074960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:36.074960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:36.074960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:36.074960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:36.074960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:36.074960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:36.074960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:36.074960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:36.074960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:36.074960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:36.074960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:36.074960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:36.075989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:36.077006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:36.077006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:36.077006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:36.077006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:36.077006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:36.077006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:36.077006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:36.077006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:36.077006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 21:53:36.077006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:36.077006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 457/901] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Jul 12 21:53:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:36.077604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad0b99-083f-43fd-a28b-86f50912c5bc None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:36.077672 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad0b99-083f-43fd-a28b-86f50912c5bc None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:36.078095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad0b99-083f-43fd-a28b-86f50912c5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:36.096901 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-8dad0b99-083f-43fd-a28b-86f50912c5bc None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:36.098093 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8dad0b99-083f-43fd-a28b-86f50912c5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:36.141612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44307c5-2d51-45b3-9ec0-4525c99bade8 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:36.180510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 458/902] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:36.186035 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad0b99-083f-43fd-a28b-86f50912c5bc None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:36.192474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21daf7-d126-48ae-9963-bbf91f550374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:36.193310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21daf7-d126-48ae-9963-bbf91f550374 None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:36.193597 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21daf7-d126-48ae-9963-bbf91f550374 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:36.193876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21daf7-d126-48ae-9963-bbf91f550374 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:36.194856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21daf7-d126-48ae-9963-bbf91f550374 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:36.197755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21daf7-d126-48ae-9963-bbf91f550374 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:36.219340 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 445/903] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Jul 12 21:53:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:53:36.220153 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21daf7-d126-48ae-9963-bbf91f550374 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:36.251460 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:36.268543 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:36.323659 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad0b99-083f-43fd-a28b-86f50912c5bc None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:36.367080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21daf7-d126-48ae-9963-bbf91f550374 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:36.370845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c8f41-c4d3-4adf-8404-49d971f31744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:36.371528 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c8f41-c4d3-4adf-8404-49d971f31744 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:36.371811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c8f41-c4d3-4adf-8404-49d971f31744 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:36.372085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c8f41-c4d3-4adf-8404-49d971f31744 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:36.373040 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c8f41-c4d3-4adf-8404-49d971f31744 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:36.373597 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c8f41-c4d3-4adf-8404-49d971f31744 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:36.374823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c8f41-c4d3-4adf-8404-49d971f31744 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:36.402911 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 459/904] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Jul 12 21:53:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:36.405661 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c8f41-c4d3-4adf-8404-49d971f31744 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:36.425466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411766aa-020b-443a-be21-a166a87dbc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:36.426338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411766aa-020b-443a-be21-a166a87dbc72 None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:36.426912 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411766aa-020b-443a-be21-a166a87dbc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:36.427211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411766aa-020b-443a-be21-a166a87dbc72 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:36.427953 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411766aa-020b-443a-be21-a166a87dbc72 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:36.429573 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411766aa-020b-443a-be21-a166a87dbc72 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:36.452299 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 446/905] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Jul 12 21:53:36 2022] POST /identity/v3/services => generated 241 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:53:36.533514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:36.538865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c8f41-c4d3-4adf-8404-49d971f31744 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:36.585422 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495cb7b-9525-4392-aa09-268b745551b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:36.586409 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495cb7b-9525-4392-aa09-268b745551b3 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:36.586957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495cb7b-9525-4392-aa09-268b745551b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:36.587424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495cb7b-9525-4392-aa09-268b745551b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:36.655945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccd50a-830b-44f3-bc0a-5aa02dfef7fe None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:36.656318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccd50a-830b-44f3-bc0a-5aa02dfef7fe None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:36.656604 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccd50a-830b-44f3-bc0a-5aa02dfef7fe None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:36.659238 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 447/906] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:36.665697 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364abc6-1550-4eb2-ab95-526cf9130011 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:36.665991 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364abc6-1550-4eb2-ab95-526cf9130011 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:36.666273 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364abc6-1550-4eb2-ab95-526cf9130011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:36.667014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495cb7b-9525-4392-aa09-268b745551b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:36.668550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495cb7b-9525-4392-aa09-268b745551b3 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:36.686879 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-a364abc6-1550-4eb2-ab95-526cf9130011 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:36.688311 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a364abc6-1550-4eb2-ab95-526cf9130011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:36.785450 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:36.792734 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 448/907] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:36.799067 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 460/908] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:53:36 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:53:36.804909 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364abc6-1550-4eb2-ab95-526cf9130011 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:36.814816 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495cb7b-9525-4392-aa09-268b745551b3 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:36.831910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:36.931096 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364abc6-1550-4eb2-ab95-526cf9130011 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:36.950326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495cb7b-9525-4392-aa09-268b745551b3 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:36.968664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4144b0f-9fde-44e5-b323-91387f79e60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:36.969746 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4144b0f-9fde-44e5-b323-91387f79e60d None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:36.970226 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4144b0f-9fde-44e5-b323-91387f79e60d None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:36.970715 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4144b0f-9fde-44e5-b323-91387f79e60d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:36.981831 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4144b0f-9fde-44e5-b323-91387f79e60d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:36.984051 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4144b0f-9fde-44e5-b323-91387f79e60d None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:36.999952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb21cea-675e-4b3f-acd8-8c223b78e6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:37.000858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb21cea-675e-4b3f-acd8-8c223b78e6de None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:37.001147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb21cea-675e-4b3f-acd8-8c223b78e6de None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:37.001424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb21cea-675e-4b3f-acd8-8c223b78e6de None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:37.002085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb21cea-675e-4b3f-acd8-8c223b78e6de None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:37.003459 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb21cea-675e-4b3f-acd8-8c223b78e6de None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:37.005331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-d4144b0f-9fde-44e5-b323-91387f79e60d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:37.005331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:37.005331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:37.005331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:37.005331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:37.005331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:37.005331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:37.005331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:37.005331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:37.005331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:37.005331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:37.005331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:37.005331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:37.005331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:37.005331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:37.006185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:37.006185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:37.006185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:37.006185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:37.006185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:37.006185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:37.006185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:37.006185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:37.006185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:37.006185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:37.006185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:37.006185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:37.006185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:37.006185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:37.006185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:37.006185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:37.006185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:37.007734 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:37.007734 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:37.007734 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:37.007734 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:37.010659 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 449/909] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:37.019209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4144b0f-9fde-44e5-b323-91387f79e60d None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:37.020561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-2eb21cea-675e-4b3f-acd8-8c223b78e6de None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 21:53:37.020561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:37.020561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:37.020561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:37.020561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:37.020561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:37.020561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:37.020561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:37.020561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 21:53:37.020561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:37.020561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:37.020561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:37.020561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:37.020561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:37.020561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:37.020561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:37.021658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:37.021658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:37.021658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:37.021658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:37.021658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:37.021658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:37.021658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:37.021658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:37.021658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:37.021658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:37.021658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:37.021658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:37.021658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:37.021658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:37.021658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:37.021658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:37.022827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:37.022827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:37.022827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:37.022827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:37.022827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:37.022827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:37.022827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:37.022827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:37.022827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:37.022827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:37.022827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:37.022827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:37.022827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:37.022827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 21:53:37.022827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:37.029006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 461/910] 10.222.0.44 () {56 vars in 1132 bytes} [Tue Jul 12 21:53:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:37.031385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb21cea-675e-4b3f-acd8-8c223b78e6de None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:37.108046 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:37.120569 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:37.143113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4144b0f-9fde-44e5-b323-91387f79e60d None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:37.182114 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7820c-9008-4337-b1ea-0d1859bd3ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:37.183217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7820c-9008-4337-b1ea-0d1859bd3ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:37.183698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7820c-9008-4337-b1ea-0d1859bd3ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:37.184168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7820c-9008-4337-b1ea-0d1859bd3ff1 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:37.185130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7820c-9008-4337-b1ea-0d1859bd3ff1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:37.185650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7820c-9008-4337-b1ea-0d1859bd3ff1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:37.186038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb21cea-675e-4b3f-acd8-8c223b78e6de None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:37.187499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7820c-9008-4337-b1ea-0d1859bd3ff1 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:37.206931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 450/911] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:37.208187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7820c-9008-4337-b1ea-0d1859bd3ff1 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:37.240085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04f146-52e0-4198-b58c-e193b31bbaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:37.241725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04f146-52e0-4198-b58c-e193b31bbaa6 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:37.242240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04f146-52e0-4198-b58c-e193b31bbaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:37.243011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04f146-52e0-4198-b58c-e193b31bbaa6 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:37.243981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04f146-52e0-4198-b58c-e193b31bbaa6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:37.244504 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04f146-52e0-4198-b58c-e193b31bbaa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:37.246611 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04f146-52e0-4198-b58c-e193b31bbaa6 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:37.253951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 462/912] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Jul 12 21:53:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:37.260899 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04f146-52e0-4198-b58c-e193b31bbaa6 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:37.339389 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7820c-9008-4337-b1ea-0d1859bd3ff1 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:37.372223 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:37.378749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328f4d8-8688-42fe-90f4-cef4bed81f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:37.379889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328f4d8-8688-42fe-90f4-cef4bed81f94 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:37.380368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328f4d8-8688-42fe-90f4-cef4bed81f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:37.384172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328f4d8-8688-42fe-90f4-cef4bed81f94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:37.388747 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04f146-52e0-4198-b58c-e193b31bbaa6 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:37.393177 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328f4d8-8688-42fe-90f4-cef4bed81f94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:37.394874 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328f4d8-8688-42fe-90f4-cef4bed81f94 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:37.410727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-3328f4d8-8688-42fe-90f4-cef4bed81f94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:37.410727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:37.410727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:37.410727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:37.410727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:37.410727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:37.410727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:37.410727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:37.410727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:37.410727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:37.410727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:37.410727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:37.410727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:37.410727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:37.410727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:37.411707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:37.411707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:37.411707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:37.411707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:37.411707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:37.411707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:37.411707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:37.411707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:37.411707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:37.411707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:37.411707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:37.411707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:37.411707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:37.411707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:37.411707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:37.411707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:37.411707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:37.412598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:37.412598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:37.412598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:37.412598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:37.412598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:37.412598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:37.412598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:37.412598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:37.417049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:37.422021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 451/913] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:37.423037 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328f4d8-8688-42fe-90f4-cef4bed81f94 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:37.433916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731196c-8e0a-4514-81d2-4fec980c5242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:37.434833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731196c-8e0a-4514-81d2-4fec980c5242 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:37.435263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731196c-8e0a-4514-81d2-4fec980c5242 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:37.435687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731196c-8e0a-4514-81d2-4fec980c5242 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:37.436547 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731196c-8e0a-4514-81d2-4fec980c5242 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:37.437040 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731196c-8e0a-4514-81d2-4fec980c5242 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:37.438868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731196c-8e0a-4514-81d2-4fec980c5242 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:37.452826 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 463/914] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Jul 12 21:53:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:37.454863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731196c-8e0a-4514-81d2-4fec980c5242 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:37.558991 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328f4d8-8688-42fe-90f4-cef4bed81f94 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:37.588126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731196c-8e0a-4514-81d2-4fec980c5242 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:37.599090 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8e396-6efd-47e2-a7c2-0e8c03dbc749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:37.599901 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8e396-6efd-47e2-a7c2-0e8c03dbc749 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:37.600341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8e396-6efd-47e2-a7c2-0e8c03dbc749 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:37.600930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8e396-6efd-47e2-a7c2-0e8c03dbc749 None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:37.601935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8e396-6efd-47e2-a7c2-0e8c03dbc749 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:37.602343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8e396-6efd-47e2-a7c2-0e8c03dbc749 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:37.603891 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8e396-6efd-47e2-a7c2-0e8c03dbc749 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:37.622964 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 452/915] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:37.624157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8e396-6efd-47e2-a7c2-0e8c03dbc749 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:37.635223 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d4871-047f-4533-8fdb-6d6372f9c8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:37.636134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d4871-047f-4533-8fdb-6d6372f9c8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:37.636566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d4871-047f-4533-8fdb-6d6372f9c8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:37.637150 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d4871-047f-4533-8fdb-6d6372f9c8d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:37.638197 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d4871-047f-4533-8fdb-6d6372f9c8d7 None admin] RBAC: Adding query filter params (interface=public, service_id=b0329101283440ffba9490cd742bf1ce, region_id=RegionOne) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:37.638730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d4871-047f-4533-8fdb-6d6372f9c8d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:37.640456 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d4871-047f-4533-8fdb-6d6372f9c8d7 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:37.652322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 464/916] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Jul 12 21:53:37 2022] GET /identity/v3/endpoints?service_id=b0329101283440ffba9490cd742bf1ce&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:37.654014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d4871-047f-4533-8fdb-6d6372f9c8d7 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:37.687072 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:37.720097 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:37.743003 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8e396-6efd-47e2-a7c2-0e8c03dbc749 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:37.781992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b554a6-6312-482d-a5eb-914bffc9c066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:37.782720 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b554a6-6312-482d-a5eb-914bffc9c066 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:37.782964 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b554a6-6312-482d-a5eb-914bffc9c066 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:37.783241 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b554a6-6312-482d-a5eb-914bffc9c066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:37.784082 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b554a6-6312-482d-a5eb-914bffc9c066 None admin] RBAC: Adding query filter params (role.id=3ee4a924b5464226903b6ad15ad59ebc, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=8d6b823cc7dc434986f77739fa0259ea) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:37.784390 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b554a6-6312-482d-a5eb-914bffc9c066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:37.785897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b554a6-6312-482d-a5eb-914bffc9c066 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:37.792072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d4871-047f-4533-8fdb-6d6372f9c8d7 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:37.798379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 453/917] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:53:37 2022] GET /identity/v3/role_assignments?user.id=8d6b823cc7dc434986f77739fa0259ea&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=3ee4a924b5464226903b6ad15ad59ebc => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:37.838632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a7d83-ebc3-4158-9b28-5fb7d763931d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:37.839687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a7d83-ebc3-4158-9b28-5fb7d763931d None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:37.840177 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a7d83-ebc3-4158-9b28-5fb7d763931d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:37.840659 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a7d83-ebc3-4158-9b28-5fb7d763931d None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:37.841633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a7d83-ebc3-4158-9b28-5fb7d763931d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:37.843253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a7d83-ebc3-4158-9b28-5fb7d763931d None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:37.852370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 465/918] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Jul 12 21:53:37 2022] GET /identity/v3/services => generated 1497 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 21:53:38.643886 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a242a-255c-44cf-ad86-c0972b1e511d None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:38.644528 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a242a-255c-44cf-ad86-c0972b1e511d None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:38.644528 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a242a-255c-44cf-ad86-c0972b1e511d None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:38.646173 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 454/919] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:38.651953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4e90f-4562-4978-b4be-da0da2afef0e None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:38.652413 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4e90f-4562-4978-b4be-da0da2afef0e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:38.652501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4e90f-4562-4978-b4be-da0da2afef0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:38.670386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-deb4e90f-4562-4978-b4be-da0da2afef0e None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:38.671582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-deb4e90f-4562-4978-b4be-da0da2afef0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:38.724826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede22bee-b2aa-4235-b0a0-67ff77669a3f None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:38.725668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede22bee-b2aa-4235-b0a0-67ff77669a3f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:38.725940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede22bee-b2aa-4235-b0a0-67ff77669a3f None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:38.727476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 455/920] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:53:38.732773 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668e1fa-6306-4940-bad1-8abce3ef0a72 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:38.733068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668e1fa-6306-4940-bad1-8abce3ef0a72 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:38.733349 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668e1fa-6306-4940-bad1-8abce3ef0a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:38.744157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 466/921] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:38.753455 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-7668e1fa-6306-4940-bad1-8abce3ef0a72 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:38.754750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-7668e1fa-6306-4940-bad1-8abce3ef0a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:38.767451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4e90f-4562-4978-b4be-da0da2afef0e None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:38.822573 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:38.834154 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 456/922] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:53:38.857708 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668e1fa-6306-4940-bad1-8abce3ef0a72 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:38.900213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4e90f-4562-4978-b4be-da0da2afef0e None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:38.926618 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:38.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4c48e-2459-4570-b7f6-dc6a48f5ed7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:38.952215 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4c48e-2459-4570-b7f6-dc6a48f5ed7f None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:38.953052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4c48e-2459-4570-b7f6-dc6a48f5ed7f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:38.953546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4c48e-2459-4570-b7f6-dc6a48f5ed7f None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:38.955518 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4c48e-2459-4570-b7f6-dc6a48f5ed7f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:38.960062 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4c48e-2459-4570-b7f6-dc6a48f5ed7f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:38.960062 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: WARNING keystone.server.flask.common [None req-e7b4c48e-2459-4570-b7f6-dc6a48f5ed7f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 12 21:53:38.982670 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.sql.core [None req-e7b4c48e-2459-4570-b7f6-dc6a48f5ed7f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 21:53:38.982670 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 12 21:53:38.982670 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [parameters: {'id': '1fa0e867dcd646f6a405db6e04fcceea', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 12 21:53:38.982670 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80218) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 21:53:38.990235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-e7b4c48e-2459-4570-b7f6-dc6a48f5ed7f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1fa0e867dcd646f6a405db6e04fcceea.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1fa0e867dcd646f6a405db6e04fcceea. Jul 12 21:53:38.990235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:38.990235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 21:53:38.990235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 21:53:38.990235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 21:53:38.990235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 21:53:38.990235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 21:53:38.990235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 21:53:38.990235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 21:53:38.990235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application conn.query(q) Jul 12 21:53:38.990235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 21:53:38.990235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 21:53:38.990235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application result.read() Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 12 21:53:38.991642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 12 21:53:38.992620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 21:53:38.992620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application next(self.gen) Jul 12 21:53:38.992620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 21:53:38.992620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application yield resource Jul 12 21:53:38.992620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 21:53:38.992620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application next(self.gen) Jul 12 21:53:38.992620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 21:53:38.992620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 21:53:38.992620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 21:53:38.992620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 21:53:38.992620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 21:53:38.992620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise self.value Jul 12 21:53:38.992620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 21:53:38.992620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 21:53:38.992620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 21:53:38.992620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application session.commit() Jul 12 21:53:38.992620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 21:53:38.993510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 21:53:38.993510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 21:53:38.993510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 21:53:38.993510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 21:53:38.993510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self.session.flush() Jul 12 21:53:38.993510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 21:53:38.993510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 21:53:38.993510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 21:53:38.993510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 21:53:38.993510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 21:53:38.993510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application compat.raise_( Jul 12 21:53:38.993510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 21:53:38.993510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception Jul 12 21:53:38.993510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 21:53:38.993510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 21:53:38.993510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 21:53:38.993510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 21:53:38.994478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 21:53:38.994478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 21:53:38.994478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 21:53:38.994478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 21:53:38.994478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 12 21:53:38.994478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 12 21:53:38.994478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 21:53:38.994478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 21:53:38.994478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 21:53:38.994478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 21:53:38.994478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 21:53:38.994478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 21:53:38.994478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 21:53:38.994478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 21:53:38.994478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 21:53:38.995257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 21:53:38.995257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 21:53:38.995257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception Jul 12 21:53:38.995257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 21:53:38.995257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 21:53:38.995257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 21:53:38.995257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 21:53:38.995257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 21:53:38.995257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 21:53:38.995257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 21:53:38.995257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application conn.query(q) Jul 12 21:53:38.995257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 21:53:38.995257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 21:53:38.995257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 21:53:38.995257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application result.read() Jul 12 21:53:38.995257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 21:53:38.995257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 21:53:38.996095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 21:53:38.996095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 21:53:38.996095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 21:53:38.996095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 21:53:38.996095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 21:53:38.996095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 21:53:38.996095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 21:53:38.996095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 12 21:53:38.996095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [parameters: {'id': '1fa0e867dcd646f6a405db6e04fcceea', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 12 21:53:38.996095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 21:53:38.996095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:38.996095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:38.996095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:38.996095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:38.996095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 12 21:53:38.998316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 12 21:53:38.998316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 21:53:38.998316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 21:53:38.998316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 12 21:53:38.998316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:38.998316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:38.998316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:38.998316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:38.998316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:38.998316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:38.998316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:38.998316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:38.998316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:38.998316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:38.998316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:38.998316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:38.998316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:39.000599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:39.000599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 12 21:53:39.000599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 12 21:53:39.000599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:39.000599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:39.000599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 12 21:53:39.000599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 12 21:53:39.000599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1fa0e867dcd646f6a405db6e04fcceea. Jul 12 21:53:39.000599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:39.000599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 467/923] 10.222.0.44 () {60 vars in 1140 bytes} [Tue Jul 12 21:53:38 2022] POST /identity/v3/projects => generated 262 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:39.000599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4c48e-2459-4570-b7f6-dc6a48f5ed7f None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:39.013208 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668e1fa-6306-4940-bad1-8abce3ef0a72 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:39.064123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340a58b-2698-4a7d-aee7-1d0113f27d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:39.065694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340a58b-2698-4a7d-aee7-1d0113f27d6d None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:39.066881 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340a58b-2698-4a7d-aee7-1d0113f27d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:39.067902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340a58b-2698-4a7d-aee7-1d0113f27d6d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:39.069771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5340a58b-2698-4a7d-aee7-1d0113f27d6d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:39.072480 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5340a58b-2698-4a7d-aee7-1d0113f27d6d None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:39.089103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-5340a58b-2698-4a7d-aee7-1d0113f27d6d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 21:53:39.089103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:39.089103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:39.089103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:39.089103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:39.089103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:39.089103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:39.089103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:39.089103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 21:53:39.089103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:39.089103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:39.089103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:39.089103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:39.089103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:39.089103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:39.089103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:39.089103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:39.090379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:39.090379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:39.090379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:39.090379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:39.090379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:39.090379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:39.090379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:39.090379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:39.090379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:39.090379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:39.090379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:39.090379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:39.090379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:39.090379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:39.090379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:39.090379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:39.091454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:39.091454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:39.091454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:39.091454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:39.091454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:39.091454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:39.091454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:39.091454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:39.091454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:39.091454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:39.091454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:39.091454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:39.091454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 21:53:39.091454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:39.098409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 457/924] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Jul 12 21:53:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:39.099384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340a58b-2698-4a7d-aee7-1d0113f27d6d None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:39.123337 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:39.126834 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4c48e-2459-4570-b7f6-dc6a48f5ed7f None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:39.163508 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c164995-ef26-49ac-aead-e947e4bd65fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:39.164313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c164995-ef26-49ac-aead-e947e4bd65fe None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:39.164601 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c164995-ef26-49ac-aead-e947e4bd65fe None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:39.164921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c164995-ef26-49ac-aead-e947e4bd65fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:39.172339 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c164995-ef26-49ac-aead-e947e4bd65fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:39.174101 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c164995-ef26-49ac-aead-e947e4bd65fe None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:39.189064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-3c164995-ef26-49ac-aead-e947e4bd65fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:39.189064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:39.189064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:39.189064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:39.189064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:39.189064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:39.189064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:39.189064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:39.189064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:39.189064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:39.189064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:39.189064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:39.189064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:39.189064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:39.189064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:39.189965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:39.189965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:39.189965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:39.189965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:39.189965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:39.189965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:39.189965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:39.189965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:39.189965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:39.189965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:39.189965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:39.189965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:39.189965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:39.189965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:39.189965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:39.189965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:39.189965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:39.190833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:39.190833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:39.190833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:39.190833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:39.190833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:39.190833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:39.190833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:39.190833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:39.190833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 468/925] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:39.201916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c164995-ef26-49ac-aead-e947e4bd65fe None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:39.236855 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340a58b-2698-4a7d-aee7-1d0113f27d6d None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:39.250764 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:39.282002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e5822-d61c-4f8f-8a0d-c2e76c911a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:39.282995 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e5822-d61c-4f8f-8a0d-c2e76c911a44 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:39.283464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e5822-d61c-4f8f-8a0d-c2e76c911a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:39.283917 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e5822-d61c-4f8f-8a0d-c2e76c911a44 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:39.285250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e5822-d61c-4f8f-8a0d-c2e76c911a44 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:39.285740 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e5822-d61c-4f8f-8a0d-c2e76c911a44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:39.287701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e5822-d61c-4f8f-8a0d-c2e76c911a44 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:39.302306 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 458/926] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Jul 12 21:53:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:39.302846 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e5822-d61c-4f8f-8a0d-c2e76c911a44 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:39.316761 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c164995-ef26-49ac-aead-e947e4bd65fe None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:39.357842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4890d-6170-4f02-be8e-c0e488e984ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:39.359373 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4890d-6170-4f02-be8e-c0e488e984ab None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:39.359912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4890d-6170-4f02-be8e-c0e488e984ab None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:39.360792 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4890d-6170-4f02-be8e-c0e488e984ab None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:39.362053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4890d-6170-4f02-be8e-c0e488e984ab None admin] RBAC: Adding query filter params (name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:39.362570 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4890d-6170-4f02-be8e-c0e488e984ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:39.365129 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4890d-6170-4f02-be8e-c0e488e984ab None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:39.401038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 469/927] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Jul 12 21:53:39 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:39.427199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e5822-d61c-4f8f-8a0d-c2e76c911a44 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:39.470656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fe8ed-567b-4eb5-8458-66423d3bbadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:39.471674 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fe8ed-567b-4eb5-8458-66423d3bbadd None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:39.472157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fe8ed-567b-4eb5-8458-66423d3bbadd None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:39.472619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fe8ed-567b-4eb5-8458-66423d3bbadd None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:39.473553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fe8ed-567b-4eb5-8458-66423d3bbadd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:39.474044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fe8ed-567b-4eb5-8458-66423d3bbadd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:39.476196 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fe8ed-567b-4eb5-8458-66423d3bbadd None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:39.482843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 459/928] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Jul 12 21:53:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:39.489453 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4890d-6170-4f02-be8e-c0e488e984ab None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:39.515121 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:39.609925 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4890d-6170-4f02-be8e-c0e488e984ab None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:39.654877 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f19cce-2a26-4e20-ac48-912d3d2995a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:39.656296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f19cce-2a26-4e20-ac48-912d3d2995a9 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:39.656793 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f19cce-2a26-4e20-ac48-912d3d2995a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:39.657263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f19cce-2a26-4e20-ac48-912d3d2995a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:39.658246 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f19cce-2a26-4e20-ac48-912d3d2995a9 None admin] RBAC: Adding query filter params (interface=public, service_id=2ffb730c7e1849069759e69a3ee674f8, region_id=RegionOne) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:39.658743 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f19cce-2a26-4e20-ac48-912d3d2995a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:39.660306 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f19cce-2a26-4e20-ac48-912d3d2995a9 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:39.667168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 470/929] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Jul 12 21:53:39 2022] GET /identity/v3/endpoints?service_id=2ffb730c7e1849069759e69a3ee674f8&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:39.672986 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fe8ed-567b-4eb5-8458-66423d3bbadd None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:39.721572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:39.792260 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fe8ed-567b-4eb5-8458-66423d3bbadd None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:39.828354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3fdfe-8f46-42ba-9003-6b1dcaed13bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:39.829402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3fdfe-8f46-42ba-9003-6b1dcaed13bf None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:39.830229 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3fdfe-8f46-42ba-9003-6b1dcaed13bf None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:39.830758 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3fdfe-8f46-42ba-9003-6b1dcaed13bf None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:39.831629 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3fdfe-8f46-42ba-9003-6b1dcaed13bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:39.833298 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3fdfe-8f46-42ba-9003-6b1dcaed13bf None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:39.840990 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 460/930] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Jul 12 21:53:39 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 21:53:39.956410 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543bf6b-f29b-49a1-8c8e-5dbd8d2d0eab None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:39.956736 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543bf6b-f29b-49a1-8c8e-5dbd8d2d0eab None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:39.957017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543bf6b-f29b-49a1-8c8e-5dbd8d2d0eab None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:39.960829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 471/931] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:39.965020 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12608a15-f2f9-4706-9da6-b8a829d520c3 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:39.965318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12608a15-f2f9-4706-9da6-b8a829d520c3 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:39.965602 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12608a15-f2f9-4706-9da6-b8a829d520c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:39.983532 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-12608a15-f2f9-4706-9da6-b8a829d520c3 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:39.984663 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-12608a15-f2f9-4706-9da6-b8a829d520c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:40.059611 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 461/932] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:40.062306 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995251a1-53fb-49f1-aecd-435cfe3b5a78 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:40.062698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995251a1-53fb-49f1-aecd-435cfe3b5a78 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:40.063120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995251a1-53fb-49f1-aecd-435cfe3b5a78 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:40.064860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 472/933] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:40.070564 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e86cb-b494-4b77-bbb1-7e445046309e None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:40.072524 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e86cb-b494-4b77-bbb1-7e445046309e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:40.072524 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e86cb-b494-4b77-bbb1-7e445046309e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:40.082390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995251a1-53fb-49f1-aecd-435cfe3b5a78 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:40.091660 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-637e86cb-b494-4b77-bbb1-7e445046309e None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:40.093259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-637e86cb-b494-4b77-bbb1-7e445046309e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:40.124464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:40.184838 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 462/934] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:40.192371 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:40.203047 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e86cb-b494-4b77-bbb1-7e445046309e None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:40.215254 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995251a1-53fb-49f1-aecd-435cfe3b5a78 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:40.255844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6ecf0-b9bd-416d-9b01-920fdf2e1a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:40.257401 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6ecf0-b9bd-416d-9b01-920fdf2e1a46 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:40.257691 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6ecf0-b9bd-416d-9b01-920fdf2e1a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:40.257979 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6ecf0-b9bd-416d-9b01-920fdf2e1a46 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:40.269618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6ecf0-b9bd-416d-9b01-920fdf2e1a46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:40.278388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-eac6ecf0-b9bd-416d-9b01-920fdf2e1a46 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 21:53:40.278388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:40.278388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:40.278388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:40.278388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:40.278388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:40.278388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:40.278388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:40.278388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:40.278388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:40.278388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:40.278388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:40.278388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:40.278388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:40.278388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:40.278388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:40.279350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:40.279350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:40.279350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:40.279350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:40.279350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:40.279350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:40.279350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:40.279350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:40.279350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:40.279350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:40.279350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:40.279350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:40.279350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:40.279350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:40.279350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:40.279350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:40.280186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:40.280186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:40.280186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:40.280186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:40.280186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:40.280186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:40.280186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:40.280186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:40.280186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:40.280186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:40.280186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 21:53:40.280186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:40.284694 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 473/935] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Jul 12 21:53:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:40.286007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6ecf0-b9bd-416d-9b01-920fdf2e1a46 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:40.342960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e86cb-b494-4b77-bbb1-7e445046309e None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:40.392734 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22765409-cbd3-48da-930d-05b5efd700e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:40.394451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22765409-cbd3-48da-930d-05b5efd700e3 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:40.395194 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22765409-cbd3-48da-930d-05b5efd700e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:40.395687 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22765409-cbd3-48da-930d-05b5efd700e3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:40.396562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22765409-cbd3-48da-930d-05b5efd700e3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:40.396884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6ecf0-b9bd-416d-9b01-920fdf2e1a46 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:40.398426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22765409-cbd3-48da-930d-05b5efd700e3 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:40.416280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-22765409-cbd3-48da-930d-05b5efd700e3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 21:53:40.416280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:40.416280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:40.416280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:40.416280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:40.416280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:40.416280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:40.416280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:40.416280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 21:53:40.416280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:40.416280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:40.416280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:40.416280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:40.416280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:40.416280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:40.416280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:40.417316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:40.417316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:40.417316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:40.417316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:40.417316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:40.417316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:40.417316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:40.417316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:40.417316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:40.417316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:40.417316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:40.417316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:40.417316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:40.417316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:40.417316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:40.417316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:40.418145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:40.418145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:40.418145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:40.418145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:40.418145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:40.418145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:40.418145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:40.418145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:40.418145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:40.418145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:40.418145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:40.418145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:40.418145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:40.418145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 21:53:40.418145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:40.421857 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 463/936] 10.222.0.44 () {56 vars in 1132 bytes} [Tue Jul 12 21:53:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:40.428788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22765409-cbd3-48da-930d-05b5efd700e3 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:40.440731 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d808c-f0a7-42f3-8f5b-b91e7ba1b5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:40.441224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d808c-f0a7-42f3-8f5b-b91e7ba1b5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:40.441765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d808c-f0a7-42f3-8f5b-b91e7ba1b5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:40.442047 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d808c-f0a7-42f3-8f5b-b91e7ba1b5d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:40.442865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d808c-f0a7-42f3-8f5b-b91e7ba1b5d3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:40.443182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d808c-f0a7-42f3-8f5b-b91e7ba1b5d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:40.445209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d808c-f0a7-42f3-8f5b-b91e7ba1b5d3 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:40.459228 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 474/937] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Jul 12 21:53:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:40.461298 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d808c-f0a7-42f3-8f5b-b91e7ba1b5d3 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:40.469566 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:40.472258 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:40.550133 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22765409-cbd3-48da-930d-05b5efd700e3 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:40.586743 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a4b62-4460-45bb-a885-2e5f5836717d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:40.587409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a4b62-4460-45bb-a885-2e5f5836717d None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:40.587698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a4b62-4460-45bb-a885-2e5f5836717d None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:40.587982 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a4b62-4460-45bb-a885-2e5f5836717d None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:40.588699 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a4b62-4460-45bb-a885-2e5f5836717d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:40.589063 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a4b62-4460-45bb-a885-2e5f5836717d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:40.590437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a4b62-4460-45bb-a885-2e5f5836717d None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:40.600815 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 464/938] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Jul 12 21:53:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:40.602010 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d808c-f0a7-42f3-8f5b-b91e7ba1b5d3 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:40.603068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a4b62-4460-45bb-a885-2e5f5836717d None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:40.645471 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87f278-1541-438e-9042-fdc750649f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:40.646341 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87f278-1541-438e-9042-fdc750649f35 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:40.646625 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87f278-1541-438e-9042-fdc750649f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:40.647491 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87f278-1541-438e-9042-fdc750649f35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:40.659891 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87f278-1541-438e-9042-fdc750649f35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:40.661515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87f278-1541-438e-9042-fdc750649f35 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:40.681606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-9b87f278-1541-438e-9042-fdc750649f35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:40.681606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:40.681606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:40.681606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:40.681606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:40.681606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:40.681606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:40.681606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:40.681606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:40.681606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:40.681606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:40.681606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:40.681606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:40.681606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:40.681606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:40.682599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:40.682599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:40.682599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:40.682599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:40.682599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:40.682599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:40.682599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:40.682599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:40.682599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:40.682599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:40.682599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:40.682599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:40.682599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:40.682599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:40.682599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:40.682599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:40.682599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:40.683506 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:40.683506 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:40.683506 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:40.683506 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:40.687551 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 475/939] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:40.688951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87f278-1541-438e-9042-fdc750649f35 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:40.728164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:40.741315 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a4b62-4460-45bb-a885-2e5f5836717d None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:40.757181 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:40.790817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edfba12-0170-4fa3-ac37-c58036b461af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:40.791486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edfba12-0170-4fa3-ac37-c58036b461af None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:40.791771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edfba12-0170-4fa3-ac37-c58036b461af None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:40.792051 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edfba12-0170-4fa3-ac37-c58036b461af None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:40.792785 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edfba12-0170-4fa3-ac37-c58036b461af None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:40.793102 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edfba12-0170-4fa3-ac37-c58036b461af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:40.794487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edfba12-0170-4fa3-ac37-c58036b461af None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:40.805374 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 465/940] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Jul 12 21:53:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:40.806346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edfba12-0170-4fa3-ac37-c58036b461af None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:40.817484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87f278-1541-438e-9042-fdc750649f35 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:40.856761 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99600358-7744-43b2-b2d1-289f8960eff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:40.856971 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99600358-7744-43b2-b2d1-289f8960eff3 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:40.857183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99600358-7744-43b2-b2d1-289f8960eff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:40.857522 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99600358-7744-43b2-b2d1-289f8960eff3 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:40.858254 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99600358-7744-43b2-b2d1-289f8960eff3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:40.858613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99600358-7744-43b2-b2d1-289f8960eff3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:40.860001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99600358-7744-43b2-b2d1-289f8960eff3 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:40.878850 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 476/941] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:40.880200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99600358-7744-43b2-b2d1-289f8960eff3 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:40.946256 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edfba12-0170-4fa3-ac37-c58036b461af None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:40.988199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0578336-ebea-45aa-bb85-639f3495cf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:40.989206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0578336-ebea-45aa-bb85-639f3495cf3f None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:40.989658 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0578336-ebea-45aa-bb85-639f3495cf3f None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:40.990107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0578336-ebea-45aa-bb85-639f3495cf3f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:40.991588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0578336-ebea-45aa-bb85-639f3495cf3f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:40.993266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0578336-ebea-45aa-bb85-639f3495cf3f None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:41.000221 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99600358-7744-43b2-b2d1-289f8960eff3 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:41.020797 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 466/942] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Jul 12 21:53:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:53:41.031056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:41.058918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305f5f1-948c-4ae2-ab83-a1295e6e1d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:41.059976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305f5f1-948c-4ae2-ab83-a1295e6e1d63 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:41.060474 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305f5f1-948c-4ae2-ab83-a1295e6e1d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:41.061001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305f5f1-948c-4ae2-ab83-a1295e6e1d63 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:41.070641 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305f5f1-948c-4ae2-ab83-a1295e6e1d63 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:41.072445 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305f5f1-948c-4ae2-ab83-a1295e6e1d63 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:41.090216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-1305f5f1-948c-4ae2-ab83-a1295e6e1d63 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 21:53:41.090216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:41.090216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:41.090216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:41.090216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:41.090216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:41.090216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:41.090216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:41.090216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:41.090216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:41.090216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:41.090216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:41.090216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:41.090216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:41.090216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:41.090216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:41.091434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:41.092609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:41.092609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:41.092609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:41.092609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:41.092609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:41.092609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:41.092609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:41.092609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:41.092609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 21:53:41.092609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:41.100892 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 477/943] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Jul 12 21:53:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:41.103158 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0578336-ebea-45aa-bb85-639f3495cf3f None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:41.118694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:41.220253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0578336-ebea-45aa-bb85-639f3495cf3f None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:41.257083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39390d15-cf7e-490d-94f7-bc3c1ce241f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:41.257811 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39390d15-cf7e-490d-94f7-bc3c1ce241f9 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:41.258099 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39390d15-cf7e-490d-94f7-bc3c1ce241f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:41.258381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39390d15-cf7e-490d-94f7-bc3c1ce241f9 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:41.259231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39390d15-cf7e-490d-94f7-bc3c1ce241f9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:41.259544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39390d15-cf7e-490d-94f7-bc3c1ce241f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:41.261006 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39390d15-cf7e-490d-94f7-bc3c1ce241f9 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:41.282202 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 467/944] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Jul 12 21:53:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:41.287226 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305f5f1-948c-4ae2-ab83-a1295e6e1d63 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:41.393519 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305f5f1-948c-4ae2-ab83-a1295e6e1d63 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:41.430057 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64195c86-5a10-47a0-b6c6-89bc6fb0b2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:41.430857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64195c86-5a10-47a0-b6c6-89bc6fb0b2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:41.431205 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64195c86-5a10-47a0-b6c6-89bc6fb0b2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:41.431484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64195c86-5a10-47a0-b6c6-89bc6fb0b2f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:41.442534 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64195c86-5a10-47a0-b6c6-89bc6fb0b2f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:41.444615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64195c86-5a10-47a0-b6c6-89bc6fb0b2f4 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:41.463362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-64195c86-5a10-47a0-b6c6-89bc6fb0b2f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:41.463362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:41.463362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:41.463362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:41.463362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:41.463362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:41.463362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:41.463362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:41.463362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:41.463362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:41.463362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:41.463362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:41.463362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:41.463362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:41.463362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:41.464299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:41.464299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:41.464299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:41.464299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:41.464299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:41.464299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:41.464299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:41.464299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:41.464299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:41.464299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:41.464299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:41.464299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:41.464299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:41.464299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:41.464299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:41.464299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:41.464299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:41.465196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:41.465196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:41.465196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:41.465196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:41.468923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 478/945] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:41.471139 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39390d15-cf7e-490d-94f7-bc3c1ce241f9 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:41.542360 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:41.580244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39390d15-cf7e-490d-94f7-bc3c1ce241f9 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:41.586494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa5ec7-1702-4989-8624-687f5e320749 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:41.586793 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa5ec7-1702-4989-8624-687f5e320749 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:41.587051 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa5ec7-1702-4989-8624-687f5e320749 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:41.588725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 479/946] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:41.593836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2c803-8368-4c42-a67e-b0f11a0273a7 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:41.594131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2c803-8368-4c42-a67e-b0f11a0273a7 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:41.594415 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2c803-8368-4c42-a67e-b0f11a0273a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:41.617969 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-87d2c803-8368-4c42-a67e-b0f11a0273a7 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:41.618874 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cb759-639e-4bc1-96ff-ee0091632e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:41.619519 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-87d2c803-8368-4c42-a67e-b0f11a0273a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:41.619941 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cb759-639e-4bc1-96ff-ee0091632e7c None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:41.620417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cb759-639e-4bc1-96ff-ee0091632e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:41.620906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cb759-639e-4bc1-96ff-ee0091632e7c None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:41.621960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cb759-639e-4bc1-96ff-ee0091632e7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:41.622669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cb759-639e-4bc1-96ff-ee0091632e7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:41.624190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cb759-639e-4bc1-96ff-ee0091632e7c None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:41.642747 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 468/947] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:41.644088 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cb759-639e-4bc1-96ff-ee0091632e7c None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:41.644882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:41.715641 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 480/948] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:41.743152 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2c803-8368-4c42-a67e-b0f11a0273a7 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:41.775359 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cb759-639e-4bc1-96ff-ee0091632e7c None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:41.813161 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:41.824471 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666c07f-cb4f-4b9f-b836-4fde2f1a2e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:41.825690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666c07f-cb4f-4b9f-b836-4fde2f1a2e50 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:41.826841 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666c07f-cb4f-4b9f-b836-4fde2f1a2e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:41.827333 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666c07f-cb4f-4b9f-b836-4fde2f1a2e50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:41.837464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666c07f-cb4f-4b9f-b836-4fde2f1a2e50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:41.839931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666c07f-cb4f-4b9f-b836-4fde2f1a2e50 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:41.856001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2c803-8368-4c42-a67e-b0f11a0273a7 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:41.859417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-0666c07f-cb4f-4b9f-b836-4fde2f1a2e50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:41.859417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:41.859417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:41.859417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:41.859417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:41.859417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:41.859417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:41.859417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:41.859417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:41.859417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:41.859417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:41.859417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:41.859417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:41.859417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:41.859417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:41.860362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:41.860362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:41.860362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:41.860362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:41.860362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:41.860362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:41.860362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:41.860362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:41.860362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:41.860362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:41.860362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:41.860362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:41.860362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:41.860362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:41.860362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:41.860362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:41.860362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:41.861296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:41.861296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:41.861296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:41.861296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:41.861296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:41.861296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:41.861296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:41.861296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:41.864803 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 469/949] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:41.870117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666c07f-cb4f-4b9f-b836-4fde2f1a2e50 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:41.904054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef7a9d-3a82-4cd1-b2a7-ca5d5d727d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:41.904682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef7a9d-3a82-4cd1-b2a7-ca5d5d727d0b None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:41.905010 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef7a9d-3a82-4cd1-b2a7-ca5d5d727d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:41.905604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef7a9d-3a82-4cd1-b2a7-ca5d5d727d0b None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:41.906166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef7a9d-3a82-4cd1-b2a7-ca5d5d727d0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:41.907656 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef7a9d-3a82-4cd1-b2a7-ca5d5d727d0b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:41.924869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.sql.core [None req-96ef7a9d-3a82-4cd1-b2a7-ca5d5d727d0b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 21:53:41.924869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 12 21:53:41.924869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [parameters: {'user_id': '3e35a325a11844a680b120ae5ad7541b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 12 21:53:41.924869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80218) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 21:53:41.934505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-96ef7a9d-3a82-4cd1-b2a7-ca5d5d727d0b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 12 21:53:41.934505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:41.934505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 21:53:41.934505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 21:53:41.934505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 21:53:41.934505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 21:53:41.934505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 21:53:41.934505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 21:53:41.934505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 21:53:41.934505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application conn.query(q) Jul 12 21:53:41.934505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 21:53:41.934505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 21:53:41.934505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 21:53:41.934505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application result.read() Jul 12 21:53:41.934505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 21:53:41.935414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application next(self.gen) Jul 12 21:53:41.936348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 21:53:41.936348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application yield resource Jul 12 21:53:41.936348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 21:53:41.936348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application next(self.gen) Jul 12 21:53:41.936348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 21:53:41.936348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 21:53:41.936348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 21:53:41.936348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 21:53:41.936348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 21:53:41.936348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise self.value Jul 12 21:53:41.936348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 21:53:41.936348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 21:53:41.936348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 21:53:41.936348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application session.commit() Jul 12 21:53:41.936348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 21:53:41.936348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 21:53:41.936348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 21:53:41.937767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 21:53:41.937767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 21:53:41.937767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self.session.flush() Jul 12 21:53:41.937767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 21:53:41.937767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 21:53:41.937767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 21:53:41.937767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 21:53:41.937767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 21:53:41.937767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application compat.raise_( Jul 12 21:53:41.937767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 21:53:41.937767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception Jul 12 21:53:41.937767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 21:53:41.937767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 21:53:41.937767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 21:53:41.937767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 21:53:41.937767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 21:53:41.937767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 21:53:41.938634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 21:53:41.938634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 21:53:41.938634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 12 21:53:41.938634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 12 21:53:41.938634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 21:53:41.938634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 21:53:41.938634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 21:53:41.938634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 21:53:41.938634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 21:53:41.938634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 21:53:41.938634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 21:53:41.938634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 21:53:41.938634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 21:53:41.938634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 21:53:41.938634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 21:53:41.938634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception Jul 12 21:53:41.939622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 21:53:41.939622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 21:53:41.939622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 21:53:41.939622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 21:53:41.939622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 21:53:41.939622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 21:53:41.939622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 21:53:41.939622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application conn.query(q) Jul 12 21:53:41.939622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 21:53:41.939622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 21:53:41.939622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 21:53:41.939622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application result.read() Jul 12 21:53:41.939622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 21:53:41.939622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 21:53:41.939622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 21:53:41.939622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 21:53:41.940479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 21:53:41.940479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 21:53:41.940479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 21:53:41.940479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 21:53:41.940479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 21:53:41.940479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 12 21:53:41.940479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [parameters: {'user_id': '3e35a325a11844a680b120ae5ad7541b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 12 21:53:41.940479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 21:53:41.940479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:41.940479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:41.940479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:41.940479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:41.940479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:41.940479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:41.940479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:41.942857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:41.942857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:41.942857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:41.942857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:41.942857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:41.942857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:41.942857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:41.942857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 12 21:53:41.942857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 12 21:53:41.942857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:41.942857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:41.942857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:41.942857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:41.942857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:41.942857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:41.942857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 12 21:53:41.942857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 12 21:53:41.945096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 12 21:53:41.945096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 12 21:53:41.945096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 21:53:41.945096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 21:53:41.945096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 12 21:53:41.945096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:41.945096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 481/950] 10.222.0.44 () {60 vars in 1131 bytes} [Tue Jul 12 21:53:41 2022] POST /identity/v3/users => generated 160 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:41.945096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef7a9d-3a82-4cd1-b2a7-ca5d5d727d0b None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:41.971338 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:42.000339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666c07f-cb4f-4b9f-b836-4fde2f1a2e50 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:42.037068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce82da-34af-4a59-bcd4-0ddc2d669f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:42.038529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce82da-34af-4a59-bcd4-0ddc2d669f52 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:42.038901 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce82da-34af-4a59-bcd4-0ddc2d669f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:42.039294 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce82da-34af-4a59-bcd4-0ddc2d669f52 None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:42.040106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce82da-34af-4a59-bcd4-0ddc2d669f52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:42.040412 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce82da-34af-4a59-bcd4-0ddc2d669f52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:42.041953 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce82da-34af-4a59-bcd4-0ddc2d669f52 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:42.054809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 470/951] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:42.061038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cbcf6-2338-4b1e-abe5-b8f2c95c15af None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:42.061335 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cbcf6-2338-4b1e-abe5-b8f2c95c15af None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:42.061651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cbcf6-2338-4b1e-abe5-b8f2c95c15af None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:42.063058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 471/952] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:42.063741 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cbcf6-2338-4b1e-abe5-b8f2c95c15af None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:42.071409 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef7a9d-3a82-4cd1-b2a7-ca5d5d727d0b None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:42.111691 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45516c-e30c-4b65-8064-1eeab05b3131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:42.112599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45516c-e30c-4b65-8064-1eeab05b3131 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:42.112975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45516c-e30c-4b65-8064-1eeab05b3131 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:42.113317 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45516c-e30c-4b65-8064-1eeab05b3131 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:42.118174 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:42.122604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45516c-e30c-4b65-8064-1eeab05b3131 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:42.124342 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45516c-e30c-4b65-8064-1eeab05b3131 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:42.139165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-2b45516c-e30c-4b65-8064-1eeab05b3131 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 21:53:42.139165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:42.139165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:42.139165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:42.139165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:42.139165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:42.139165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:42.139165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:42.139165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:42.139165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:42.139165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:42.139165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:42.139165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:42.139165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:42.139165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:42.139165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:42.140340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:42.141385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:42.141385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:42.141385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:42.141385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:42.141385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:42.141385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:42.141385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:42.141385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:42.141385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 21:53:42.141385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:42.141385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 482/953] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:53:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:42.142074 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334448aa-5070-407d-bb25-59c85a00697d None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:42.142312 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334448aa-5070-407d-bb25-59c85a00697d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:42.142581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334448aa-5070-407d-bb25-59c85a00697d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:42.159987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-334448aa-5070-407d-bb25-59c85a00697d None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:42.161454 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-334448aa-5070-407d-bb25-59c85a00697d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:42.183000 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cbcf6-2338-4b1e-abe5-b8f2c95c15af None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:42.221610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0fcff-e3b2-438d-b939-b5a25b206d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:42.222503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0fcff-e3b2-438d-b939-b5a25b206d60 None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:42.222790 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0fcff-e3b2-438d-b939-b5a25b206d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:42.223074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0fcff-e3b2-438d-b939-b5a25b206d60 None admin] PATH_INFO: `/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/8d6b823cc7dc434986f77739fa0259ea/roles/3ee4a924b5464226903b6ad15ad59ebc` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:42.225402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f0fcff-e3b2-438d-b939-b5a25b206d60 None admin] RBAC: Authorizing `identity:create_grant(project_id=459e4743ff3043f69408ca2095be3bd0, user_id=8d6b823cc7dc434986f77739fa0259ea, role_id=3ee4a924b5464226903b6ad15ad59ebc)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:42.226908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f0fcff-e3b2-438d-b939-b5a25b206d60 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:42.233428 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 483/954] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:42.248738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334448aa-5070-407d-bb25-59c85a00697d None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:42.249192 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 472/955] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:53:42 2022] PUT /identity/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/8d6b823cc7dc434986f77739fa0259ea/roles/3ee4a924b5464226903b6ad15ad59ebc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:53:42.255525 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:42.257831 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0fcff-e3b2-438d-b939-b5a25b206d60 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:42.374008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334448aa-5070-407d-bb25-59c85a00697d None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:42.394959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0fcff-e3b2-438d-b939-b5a25b206d60 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:42.405991 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:42.418181 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324424e-dfa7-4cd2-bdb8-cb569f8640e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:42.419426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324424e-dfa7-4cd2-bdb8-cb569f8640e2 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:42.419979 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324424e-dfa7-4cd2-bdb8-cb569f8640e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:42.420269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324424e-dfa7-4cd2-bdb8-cb569f8640e2 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:42.421204 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324424e-dfa7-4cd2-bdb8-cb569f8640e2 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:42.421514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324424e-dfa7-4cd2-bdb8-cb569f8640e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:42.422961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324424e-dfa7-4cd2-bdb8-cb569f8640e2 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:42.442938 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29552b3a-a609-4012-b1ea-8fc6c50cbc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:42.443791 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29552b3a-a609-4012-b1ea-8fc6c50cbc6f None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:42.444256 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29552b3a-a609-4012-b1ea-8fc6c50cbc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:42.444541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29552b3a-a609-4012-b1ea-8fc6c50cbc6f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:42.445248 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29552b3a-a609-4012-b1ea-8fc6c50cbc6f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:42.446753 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29552b3a-a609-4012-b1ea-8fc6c50cbc6f None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:42.460184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-29552b3a-a609-4012-b1ea-8fc6c50cbc6f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 21:53:42.460184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:42.460184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:42.460184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:42.460184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:42.460184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:42.460184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:42.460184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:42.460184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 21:53:42.460184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:42.460184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:42.460184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:42.460184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:42.460184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:42.460184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:42.460184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:42.460184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:42.461491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:42.461491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:42.461491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:42.461491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:42.461491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:42.461491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:42.461491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:42.461491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:42.461491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:42.461491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:42.461491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:42.461491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:42.461491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:42.461491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:42.461491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:42.461491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:42.462355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:42.462355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:42.462355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:42.462355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:42.462355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:42.462355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:42.462355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:42.462355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:42.462355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:42.462355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:42.462355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:42.462355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:42.462355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 21:53:42.462355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:42.462355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 473/956] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Jul 12 21:53:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:42.469190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29552b3a-a609-4012-b1ea-8fc6c50cbc6f None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:42.476815 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 484/957] 10.222.0.44 () {56 vars in 1101 bytes} [Tue Jul 12 21:53:42 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:42.588064 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29552b3a-a609-4012-b1ea-8fc6c50cbc6f None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:42.626305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee6041-2c09-4b58-b0de-6fa6b862a734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:42.626989 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee6041-2c09-4b58-b0de-6fa6b862a734 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:42.627275 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee6041-2c09-4b58-b0de-6fa6b862a734 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:42.627613 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee6041-2c09-4b58-b0de-6fa6b862a734 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:42.628348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee6041-2c09-4b58-b0de-6fa6b862a734 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:42.628658 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee6041-2c09-4b58-b0de-6fa6b862a734 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:42.630094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee6041-2c09-4b58-b0de-6fa6b862a734 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:42.637443 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 474/958] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Jul 12 21:53:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:42.643038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324424e-dfa7-4cd2-bdb8-cb569f8640e2 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:42.723767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:42.762827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324424e-dfa7-4cd2-bdb8-cb569f8640e2 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:42.802365 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad7748-70a4-486d-ac58-e474dd3b33ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:42.804029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad7748-70a4-486d-ac58-e474dd3b33ca None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:42.804546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad7748-70a4-486d-ac58-e474dd3b33ca None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:42.805057 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad7748-70a4-486d-ac58-e474dd3b33ca None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:42.805973 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad7748-70a4-486d-ac58-e474dd3b33ca None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:42.806487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad7748-70a4-486d-ac58-e474dd3b33ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:42.808080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad7748-70a4-486d-ac58-e474dd3b33ca None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:42.813621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 485/959] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Jul 12 21:53:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:42.820335 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee6041-2c09-4b58-b0de-6fa6b862a734 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:42.859206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:42.932499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee6041-2c09-4b58-b0de-6fa6b862a734 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:42.974444 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60716b58-d417-45c1-8d1c-5a03c31ed370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:42.975658 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60716b58-d417-45c1-8d1c-5a03c31ed370 None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:42.976005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60716b58-d417-45c1-8d1c-5a03c31ed370 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:42.976290 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60716b58-d417-45c1-8d1c-5a03c31ed370 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:42.977100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60716b58-d417-45c1-8d1c-5a03c31ed370 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:42.978494 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60716b58-d417-45c1-8d1c-5a03c31ed370 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:43.004458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 475/960] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Jul 12 21:53:42 2022] POST /identity/v3/endpoints => generated 349 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:53:43.294555 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8012b-26b2-4b7c-bec9-3f778f36d455 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:43.294954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8012b-26b2-4b7c-bec9-3f778f36d455 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:43.295132 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8012b-26b2-4b7c-bec9-3f778f36d455 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:43.296648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 486/961] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:43.301658 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b97397-1e3b-4404-9856-0965c9271270 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:43.301959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b97397-1e3b-4404-9856-0965c9271270 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:43.302240 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b97397-1e3b-4404-9856-0965c9271270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:43.319865 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-b2b97397-1e3b-4404-9856-0965c9271270 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:43.321106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b2b97397-1e3b-4404-9856-0965c9271270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:43.398078 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 476/962] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:43.419596 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8012b-26b2-4b7c-bec9-3f778f36d455 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:43.531294 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8012b-26b2-4b7c-bec9-3f778f36d455 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:43.561020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:43.568252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd466d3-0825-46e2-91bd-cd024f36da97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:43.569116 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd466d3-0825-46e2-91bd-cd024f36da97 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:43.569404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd466d3-0825-46e2-91bd-cd024f36da97 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:43.569684 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd466d3-0825-46e2-91bd-cd024f36da97 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:43.570345 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd466d3-0825-46e2-91bd-cd024f36da97 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:43.571723 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd466d3-0825-46e2-91bd-cd024f36da97 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:43.583722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-9cd466d3-0825-46e2-91bd-cd024f36da97 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 21:53:43.583722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:43.583722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:43.583722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:43.583722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:43.583722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:43.583722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:43.583722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:43.583722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 21:53:43.583722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:43.583722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:43.583722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:43.583722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:43.583722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:43.583722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:43.583722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:43.583722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:43.586054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:43.586054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:43.586054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:43.586054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:43.586054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:43.586054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:43.586054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:43.586054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:43.586054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:43.586054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:43.586054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:43.586054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:43.586054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:43.586054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:43.586054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:43.586054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:43.587209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:43.587209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:43.587209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:43.587209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:43.587209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:43.587209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:43.587209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:43.587209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:43.587209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:43.587209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:43.587209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:43.587209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:43.587209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 21:53:43.587209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:43.589118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 487/963] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:43.591293 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b97397-1e3b-4404-9856-0965c9271270 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:43.610131 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:43.694870 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b97397-1e3b-4404-9856-0965c9271270 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:43.731440 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87800aed-0a38-4fd8-b218-632529865214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:43.732110 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87800aed-0a38-4fd8-b218-632529865214 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:43.732393 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87800aed-0a38-4fd8-b218-632529865214 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:43.732671 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87800aed-0a38-4fd8-b218-632529865214 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:43.733409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87800aed-0a38-4fd8-b218-632529865214 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:43.733721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87800aed-0a38-4fd8-b218-632529865214 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:43.735110 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87800aed-0a38-4fd8-b218-632529865214 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:43.740146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 477/964] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Jul 12 21:53:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:43.745551 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd466d3-0825-46e2-91bd-cd024f36da97 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:43.848151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd466d3-0825-46e2-91bd-cd024f36da97 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:43.882797 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab16eb3-9bfb-4a36-8c5e-c9331243a763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:43.885007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab16eb3-9bfb-4a36-8c5e-c9331243a763 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:43.885007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab16eb3-9bfb-4a36-8c5e-c9331243a763 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:43.885007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab16eb3-9bfb-4a36-8c5e-c9331243a763 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:43.885007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab16eb3-9bfb-4a36-8c5e-c9331243a763 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:43.885007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab16eb3-9bfb-4a36-8c5e-c9331243a763 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:43.886409 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab16eb3-9bfb-4a36-8c5e-c9331243a763 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:43.897695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 488/965] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Jul 12 21:53:43 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:44.369852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e5269-4d77-474a-aba1-b6e023ffc898 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:44.370380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e5269-4d77-474a-aba1-b6e023ffc898 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:44.370649 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e5269-4d77-474a-aba1-b6e023ffc898 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:44.372003 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 478/966] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:44.377359 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b413252-4ccd-4941-87a7-f6f7c3096027 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:44.377653 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b413252-4ccd-4941-87a7-f6f7c3096027 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:44.377931 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b413252-4ccd-4941-87a7-f6f7c3096027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:44.394942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-7b413252-4ccd-4941-87a7-f6f7c3096027 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:44.396113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-7b413252-4ccd-4941-87a7-f6f7c3096027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:44.450087 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:44.465977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 489/967] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:44.488700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e5269-4d77-474a-aba1-b6e023ffc898 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:44.574745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:44.594914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e5269-4d77-474a-aba1-b6e023ffc898 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:44.629767 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca5ba4-bdfb-4b98-9603-066785897914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:44.630457 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5ba4-bdfb-4b98-9603-066785897914 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:44.630793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5ba4-bdfb-4b98-9603-066785897914 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:44.631053 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5ba4-bdfb-4b98-9603-066785897914 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:44.642459 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca5ba4-bdfb-4b98-9603-066785897914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:44.650970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-6dca5ba4-bdfb-4b98-9603-066785897914 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 21:53:44.650970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:44.650970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:44.650970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:44.650970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:44.650970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:44.650970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:44.650970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:44.650970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:44.650970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:44.650970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:44.650970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:44.650970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:44.650970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:44.650970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:44.650970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:44.652150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:44.652150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:44.652150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:44.652150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:44.652150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:44.652150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:44.652150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:44.652150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:44.652150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:44.652150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:44.652150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:44.652150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:44.652150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:44.652150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:44.652150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:44.652150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:44.653572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:44.653572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:44.653572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:44.653572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:44.653572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:44.653572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:44.653572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:44.653572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:44.653572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:44.653572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:44.653572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 21:53:44.653572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:44.657167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 479/968] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Jul 12 21:53:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:44.658655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b413252-4ccd-4941-87a7-f6f7c3096027 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:44.768345 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b413252-4ccd-4941-87a7-f6f7c3096027 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:44.802961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe79d2-c68a-4282-8c6e-097b8544fbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:44.804228 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe79d2-c68a-4282-8c6e-097b8544fbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:44.804508 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe79d2-c68a-4282-8c6e-097b8544fbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:44.804801 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe79d2-c68a-4282-8c6e-097b8544fbf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:44.805542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe79d2-c68a-4282-8c6e-097b8544fbf4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:44.805847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe79d2-c68a-4282-8c6e-097b8544fbf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:44.807215 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe79d2-c68a-4282-8c6e-097b8544fbf4 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:44.816211 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 490/969] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Jul 12 21:53:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:44.821844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca5ba4-bdfb-4b98-9603-066785897914 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:44.923823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca5ba4-bdfb-4b98-9603-066785897914 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:44.963472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7b99f-b8be-4c6d-817a-9da18dfb5874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:44.963472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7b99f-b8be-4c6d-817a-9da18dfb5874 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:44.963472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7b99f-b8be-4c6d-817a-9da18dfb5874 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:44.963472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7b99f-b8be-4c6d-817a-9da18dfb5874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:44.975411 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c7b99f-b8be-4c6d-817a-9da18dfb5874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:44.976994 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c7b99f-b8be-4c6d-817a-9da18dfb5874 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:44.978857 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:44.996603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-65c7b99f-b8be-4c6d-817a-9da18dfb5874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:44.996603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:44.996603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:44.996603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:44.996603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:44.996603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:44.996603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:44.996603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:44.996603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:44.996603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:44.996603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:44.996603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:44.996603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:44.996603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:44.996603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:44.997537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:44.997537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:44.997537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:44.997537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:44.997537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:44.997537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:44.997537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:44.997537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:44.997537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:44.997537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:44.997537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:44.997537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:44.997537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:44.997537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:44.997537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:44.997537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:44.997537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:45.003152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:45.003152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:45.003152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:45.003152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:45.003152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 480/970] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:45.003620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe79d2-c68a-4282-8c6e-097b8544fbf4 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:45.074991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:45.077714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc7560-68dc-4741-8d16-056b844e2e9a None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:45.078007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc7560-68dc-4741-8d16-056b844e2e9a None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:45.078896 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc7560-68dc-4741-8d16-056b844e2e9a None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:45.079862 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 481/971] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:45.085066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd246f-d023-475b-b478-febd08fddb1d None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:45.085415 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd246f-d023-475b-b478-febd08fddb1d None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:45.085575 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd246f-d023-475b-b478-febd08fddb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:45.102966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-3cdd246f-d023-475b-b478-febd08fddb1d None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:45.104182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-3cdd246f-d023-475b-b478-febd08fddb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:45.134452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe79d2-c68a-4282-8c6e-097b8544fbf4 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:45.169404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b0992-3651-42ce-abfa-c3a24770b4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:45.170218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b0992-3651-42ce-abfa-c3a24770b4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:45.170792 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b0992-3651-42ce-abfa-c3a24770b4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:45.171105 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b0992-3651-42ce-abfa-c3a24770b4c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:45.171965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b0992-3651-42ce-abfa-c3a24770b4c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:45.172271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b0992-3651-42ce-abfa-c3a24770b4c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:45.173662 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b0992-3651-42ce-abfa-c3a24770b4c0 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:45.180071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 482/972] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:53:45.192788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 491/973] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:45.195330 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd246f-d023-475b-b478-febd08fddb1d None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:45.203524 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b0992-3651-42ce-abfa-c3a24770b4c0 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:45.308312 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b0992-3651-42ce-abfa-c3a24770b4c0 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:45.310495 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd246f-d023-475b-b478-febd08fddb1d None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:45.321428 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:45.336180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:45.348167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c4a32-5e60-4fb3-9ad4-5cc4ed512167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:45.348342 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819d34d-4e50-43af-94c3-63e0b00ab66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:45.349000 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819d34d-4e50-43af-94c3-63e0b00ab66f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:45.349117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c4a32-5e60-4fb3-9ad4-5cc4ed512167 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:45.349284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819d34d-4e50-43af-94c3-63e0b00ab66f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:45.349379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c4a32-5e60-4fb3-9ad4-5cc4ed512167 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:45.349548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819d34d-4e50-43af-94c3-63e0b00ab66f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:45.349644 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c4a32-5e60-4fb3-9ad4-5cc4ed512167 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:45.351008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819d34d-4e50-43af-94c3-63e0b00ab66f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:45.352389 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819d34d-4e50-43af-94c3-63e0b00ab66f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:45.360226 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c4a32-5e60-4fb3-9ad4-5cc4ed512167 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:45.361821 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c4a32-5e60-4fb3-9ad4-5cc4ed512167 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:45.364644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-4819d34d-4e50-43af-94c3-63e0b00ab66f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 21:53:45.364644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:45.364644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:45.364644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:45.364644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:45.364644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:45.364644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:45.364644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:45.364644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 21:53:45.364644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:45.364644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:45.364644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:45.364644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:45.364644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:45.364644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:45.364644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:45.364644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:45.366370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:45.366370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:45.366370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:45.366370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:45.366370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:45.366370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:45.366370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:45.366370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:45.366370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:45.366370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:45.366370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:45.366370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:45.366370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:45.366370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:45.366370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:45.366370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:45.367264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:45.367264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:45.367264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:45.367264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:45.367264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:45.367264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:45.367264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:45.367264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:45.367264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:45.367264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:45.367264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:45.367264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:45.367264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 21:53:45.367264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:45.371553 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 492/974] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Jul 12 21:53:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:45.372208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819d34d-4e50-43af-94c3-63e0b00ab66f None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:45.380383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-e49c4a32-5e60-4fb3-9ad4-5cc4ed512167 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 21:53:45.380383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:45.380383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:45.380383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:45.380383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:45.380383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:45.380383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:45.380383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:45.380383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:45.380383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:45.380383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:45.380383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:45.380383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:45.380383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:45.380383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:45.380383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:45.387800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:45.389651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:45.389651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:45.389651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:45.389651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:45.389651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:45.389651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:45.389651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:45.389651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:45.389651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 21:53:45.389651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:45.389651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 483/975] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Jul 12 21:53:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:45.391252 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c4a32-5e60-4fb3-9ad4-5cc4ed512167 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:45.482825 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819d34d-4e50-43af-94c3-63e0b00ab66f None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:45.499021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c4a32-5e60-4fb3-9ad4-5cc4ed512167 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:45.520058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907ff28-53a7-4f7e-836f-97160bc50c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:45.520740 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907ff28-53a7-4f7e-836f-97160bc50c6b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:45.521044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907ff28-53a7-4f7e-836f-97160bc50c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:45.521313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907ff28-53a7-4f7e-836f-97160bc50c6b None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:45.522074 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907ff28-53a7-4f7e-836f-97160bc50c6b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:45.522379 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907ff28-53a7-4f7e-836f-97160bc50c6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:45.523941 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907ff28-53a7-4f7e-836f-97160bc50c6b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:45.529439 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 493/976] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Jul 12 21:53:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:45.534984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907ff28-53a7-4f7e-836f-97160bc50c6b None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:45.539214 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cfb63-0fac-4aa5-ba7b-c56452581126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:45.539888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cfb63-0fac-4aa5-ba7b-c56452581126 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:45.540176 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cfb63-0fac-4aa5-ba7b-c56452581126 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:45.540456 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cfb63-0fac-4aa5-ba7b-c56452581126 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:45.541343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761cfb63-0fac-4aa5-ba7b-c56452581126 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:45.541653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761cfb63-0fac-4aa5-ba7b-c56452581126 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:45.543101 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761cfb63-0fac-4aa5-ba7b-c56452581126 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:45.562077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 484/977] 10.222.0.44 () {56 vars in 1149 bytes} [Tue Jul 12 21:53:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:45.567872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cfb63-0fac-4aa5-ba7b-c56452581126 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:45.586989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:45.614513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:45.657166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907ff28-53a7-4f7e-836f-97160bc50c6b None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:45.679150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cfb63-0fac-4aa5-ba7b-c56452581126 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:45.695001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3bf64-2043-440e-904b-3d61ad4a3181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:45.695685 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3bf64-2043-440e-904b-3d61ad4a3181 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:45.695964 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3bf64-2043-440e-904b-3d61ad4a3181 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:45.696248 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3bf64-2043-440e-904b-3d61ad4a3181 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:45.696994 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3bf64-2043-440e-904b-3d61ad4a3181 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:45.697304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3bf64-2043-440e-904b-3d61ad4a3181 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:45.698680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3bf64-2043-440e-904b-3d61ad4a3181 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:45.708639 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 494/978] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Jul 12 21:53:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:45.709808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3bf64-2043-440e-904b-3d61ad4a3181 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:45.724277 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c35df-6893-4ec1-ad1a-1d9e4447f2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:45.725701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c35df-6893-4ec1-ad1a-1d9e4447f2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:45.726356 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c35df-6893-4ec1-ad1a-1d9e4447f2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:45.726640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c35df-6893-4ec1-ad1a-1d9e4447f2f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:45.800962 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c35df-6893-4ec1-ad1a-1d9e4447f2f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:45.802433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c35df-6893-4ec1-ad1a-1d9e4447f2f6 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:45.817564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3bf64-2043-440e-904b-3d61ad4a3181 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:45.820824 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:45.847159 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:45.858071 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad5930-8919-4fc8-ad81-7aeff905e98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:45.858739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad5930-8919-4fc8-ad81-7aeff905e98c None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:45.859466 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad5930-8919-4fc8-ad81-7aeff905e98c None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:45.859885 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad5930-8919-4fc8-ad81-7aeff905e98c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:45.860734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad5930-8919-4fc8-ad81-7aeff905e98c None admin] RBAC: Adding query filter params (interface=admin, service_id=2ffb730c7e1849069759e69a3ee674f8, region_id=RegionOne) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:45.861050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad5930-8919-4fc8-ad81-7aeff905e98c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:45.862439 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad5930-8919-4fc8-ad81-7aeff905e98c None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:45.868306 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 495/979] 10.222.0.44 () {56 vars in 1249 bytes} [Tue Jul 12 21:53:45 2022] GET /identity/v3/endpoints?service_id=2ffb730c7e1849069759e69a3ee674f8&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:45.873934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad5930-8919-4fc8-ad81-7aeff905e98c None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:45.915100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 485/980] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:53:45 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:53:45.920644 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c35df-6893-4ec1-ad1a-1d9e4447f2f6 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:45.986059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad5930-8919-4fc8-ad81-7aeff905e98c None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:46.023996 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a133e-aacd-4e5e-8797-be0bb57812d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:46.024672 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a133e-aacd-4e5e-8797-be0bb57812d5 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:46.024987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a133e-aacd-4e5e-8797-be0bb57812d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:46.025271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a133e-aacd-4e5e-8797-be0bb57812d5 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:46.025965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a133e-aacd-4e5e-8797-be0bb57812d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:46.027348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a133e-aacd-4e5e-8797-be0bb57812d5 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:46.033852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 496/981] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Jul 12 21:53:45 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 21:53:46.035264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f3311-2c78-4057-a3f1-1c7b17036a57 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:46.035564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f3311-2c78-4057-a3f1-1c7b17036a57 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:46.035852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f3311-2c78-4057-a3f1-1c7b17036a57 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:46.035968 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c35df-6893-4ec1-ad1a-1d9e4447f2f6 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:46.037303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 497/982] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:46.043815 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2647b-cede-47dc-81f2-60b0aa30e2b3 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:46.044130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2647b-cede-47dc-81f2-60b0aa30e2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:46.044383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2647b-cede-47dc-81f2-60b0aa30e2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:46.063651 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-95f2647b-cede-47dc-81f2-60b0aa30e2b3 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:46.064837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-95f2647b-cede-47dc-81f2-60b0aa30e2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:46.074847 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79551e-b83b-4af3-8a24-e916c104786e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:46.075844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79551e-b83b-4af3-8a24-e916c104786e None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:46.076130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79551e-b83b-4af3-8a24-e916c104786e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:46.076406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79551e-b83b-4af3-8a24-e916c104786e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:46.087866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79551e-b83b-4af3-8a24-e916c104786e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:46.089426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79551e-b83b-4af3-8a24-e916c104786e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:46.091723 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:46.094508 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:46.111924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-7b79551e-b83b-4af3-8a24-e916c104786e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:46.111924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:46.111924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:46.111924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:46.111924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:46.111924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:46.111924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:46.111924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:46.111924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:46.111924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:46.111924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:46.111924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:46.111924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:46.111924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:46.111924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:46.113269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:46.113269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:46.113269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:46.113269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:46.113269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:46.113269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:46.113269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:46.113269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:46.113269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:46.113269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:46.113269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:46.113269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:46.113269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:46.113269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:46.113269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:46.113269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:46.113269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:46.114194 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:46.114194 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:46.114194 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:46.114194 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:46.116914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 486/983] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:46.120000 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79551e-b83b-4af3-8a24-e916c104786e None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:46.152119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 498/984] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:53:46.175147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2647b-cede-47dc-81f2-60b0aa30e2b3 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:46.242762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79551e-b83b-4af3-8a24-e916c104786e None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:46.278449 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee469e-aab3-4fbd-8ffe-6cd84b77fb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:46.279117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee469e-aab3-4fbd-8ffe-6cd84b77fb07 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:46.279455 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee469e-aab3-4fbd-8ffe-6cd84b77fb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:46.279736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee469e-aab3-4fbd-8ffe-6cd84b77fb07 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:46.280486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee469e-aab3-4fbd-8ffe-6cd84b77fb07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:46.280822 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee469e-aab3-4fbd-8ffe-6cd84b77fb07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:46.282210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee469e-aab3-4fbd-8ffe-6cd84b77fb07 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:46.285368 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2647b-cede-47dc-81f2-60b0aa30e2b3 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:46.295829 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 487/985] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:46.301534 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee469e-aab3-4fbd-8ffe-6cd84b77fb07 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:46.326932 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8babe-a80f-4263-ba95-da8196ebf108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:46.327520 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8babe-a80f-4263-ba95-da8196ebf108 None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:46.327814 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8babe-a80f-4263-ba95-da8196ebf108 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:46.328091 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8babe-a80f-4263-ba95-da8196ebf108 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:46.328847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8babe-a80f-4263-ba95-da8196ebf108 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:46.330444 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8babe-a80f-4263-ba95-da8196ebf108 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:46.345774 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 499/986] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Jul 12 21:53:46 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 21:53:46.372701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:46.429288 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee469e-aab3-4fbd-8ffe-6cd84b77fb07 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:46.474809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154301e-596b-4497-bdaf-ad4840abc6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:46.475702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154301e-596b-4497-bdaf-ad4840abc6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:46.475985 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154301e-596b-4497-bdaf-ad4840abc6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:46.476261 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154301e-596b-4497-bdaf-ad4840abc6d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:46.484036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154301e-596b-4497-bdaf-ad4840abc6d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:46.485549 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154301e-596b-4497-bdaf-ad4840abc6d9 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:46.501555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-e154301e-596b-4497-bdaf-ad4840abc6d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:46.501555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:46.501555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:46.501555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:46.501555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:46.501555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:46.501555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:46.501555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:46.501555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:46.501555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:46.501555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:46.501555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:46.501555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:46.501555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:46.501555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:46.502517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:46.502517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:46.502517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:46.502517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:46.502517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:46.502517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:46.502517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:46.502517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:46.502517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:46.502517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:46.502517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:46.502517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:46.502517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:46.502517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:46.502517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:46.502517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:46.502517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:46.503406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:46.503406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:46.503406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:46.503406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:46.503406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:46.503406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:46.503406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:46.503406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:46.503406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 488/987] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:46.511456 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8babe-a80f-4263-ba95-da8196ebf108 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:46.551523 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:46.644073 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8babe-a80f-4263-ba95-da8196ebf108 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:46.687876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fc04a-cb7a-4407-9600-4d5e0c9cae0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:46.688573 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fc04a-cb7a-4407-9600-4d5e0c9cae0b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:46.690976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fc04a-cb7a-4407-9600-4d5e0c9cae0b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:46.690976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fc04a-cb7a-4407-9600-4d5e0c9cae0b None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:46.690976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fc04a-cb7a-4407-9600-4d5e0c9cae0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:46.690976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fc04a-cb7a-4407-9600-4d5e0c9cae0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:46.691962 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fc04a-cb7a-4407-9600-4d5e0c9cae0b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:46.707440 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 500/988] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:46.713845 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154301e-596b-4497-bdaf-ad4840abc6d9 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:46.821819 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154301e-596b-4497-bdaf-ad4840abc6d9 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:46.833910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:46.873994 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e21d8-67dc-4633-ab65-530dc98cfbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:46.874680 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e21d8-67dc-4633-ab65-530dc98cfbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:46.874965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e21d8-67dc-4633-ab65-530dc98cfbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:46.875246 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e21d8-67dc-4633-ab65-530dc98cfbf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:46.876096 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e21d8-67dc-4633-ab65-530dc98cfbf0 None admin] RBAC: Adding query filter params (role.id=3ee4a924b5464226903b6ad15ad59ebc, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=8d6b823cc7dc434986f77739fa0259ea) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:46.876410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e21d8-67dc-4633-ab65-530dc98cfbf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:46.877889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e21d8-67dc-4633-ab65-530dc98cfbf0 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:46.885755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 489/989] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:53:46 2022] GET /identity/v3/role_assignments?user.id=8d6b823cc7dc434986f77739fa0259ea&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=3ee4a924b5464226903b6ad15ad59ebc => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:48.181942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406db4b3-0397-4dd6-a195-e8bba1f3d98e None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:48.182469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406db4b3-0397-4dd6-a195-e8bba1f3d98e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:48.182469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406db4b3-0397-4dd6-a195-e8bba1f3d98e None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:48.184754 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 501/990] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:48.189490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dc225-2acd-49c7-a2a2-adfb1c25f630 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:48.189975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dc225-2acd-49c7-a2a2-adfb1c25f630 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:48.190056 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dc225-2acd-49c7-a2a2-adfb1c25f630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:48.208751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-494dc225-2acd-49c7-a2a2-adfb1c25f630 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:48.210646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-494dc225-2acd-49c7-a2a2-adfb1c25f630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:48.284742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 490/991] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:48.307513 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406db4b3-0397-4dd6-a195-e8bba1f3d98e None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:48.397375 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:48.423243 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406db4b3-0397-4dd6-a195-e8bba1f3d98e None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:48.473923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbdf64-c6c9-484f-b8ac-c503e306fdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:48.475894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbdf64-c6c9-484f-b8ac-c503e306fdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:48.476324 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbdf64-c6c9-484f-b8ac-c503e306fdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:48.476769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbdf64-c6c9-484f-b8ac-c503e306fdc5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:48.478448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbdf64-c6c9-484f-b8ac-c503e306fdc5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:48.481021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbdf64-c6c9-484f-b8ac-c503e306fdc5 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:48.494591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ae871-7152-41a3-aae5-28f91c8c820f None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:48.494893 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ae871-7152-41a3-aae5-28f91c8c820f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:48.495120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-bedbdf64-c6c9-484f-b8ac-c503e306fdc5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 21:53:48.495120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:48.495120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:48.495120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:48.495120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:48.495120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:48.495120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:48.495120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:48.495120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 21:53:48.495120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:48.495120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:48.495120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:48.495120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:48.495120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:48.495120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:48.495120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:48.495120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:48.496058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:48.496058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:48.496058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:48.496058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:48.496058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:48.496058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:48.496058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:48.496058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:48.496058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:48.496058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:48.496058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:48.496058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:48.496058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:48.496058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:48.496058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:48.496058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:48.497490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:48.497490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:48.497490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:48.497490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:48.497490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:48.497490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:48.497490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:48.497490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:48.497490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:48.497490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:48.497490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:48.497490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:48.497490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 21:53:48.497490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:48.498471 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ae871-7152-41a3-aae5-28f91c8c820f None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:48.498471 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 491/992] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:48.501054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 502/993] 10.222.0.44 () {56 vars in 1126 bytes} [Tue Jul 12 21:53:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:48.501816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195c6bd-d4e6-4c0d-b761-004f6327c7da None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:48.502106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195c6bd-d4e6-4c0d-b761-004f6327c7da None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:48.502386 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195c6bd-d4e6-4c0d-b761-004f6327c7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:48.502629 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbdf64-c6c9-484f-b8ac-c503e306fdc5 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:48.519384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-d195c6bd-d4e6-4c0d-b761-004f6327c7da None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:48.520570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d195c6bd-d4e6-4c0d-b761-004f6327c7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:48.595901 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 492/994] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:53:48.615132 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbdf64-c6c9-484f-b8ac-c503e306fdc5 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:48.619594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195c6bd-d4e6-4c0d-b761-004f6327c7da None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:48.654826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:48.656360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e7e37-13b5-43e7-a434-954075d71b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:48.657468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e7e37-13b5-43e7-a434-954075d71b20 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:48.658164 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e7e37-13b5-43e7-a434-954075d71b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:48.658356 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e7e37-13b5-43e7-a434-954075d71b20 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:48.659598 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e7e37-13b5-43e7-a434-954075d71b20 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:48.660135 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e7e37-13b5-43e7-a434-954075d71b20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:48.662084 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e7e37-13b5-43e7-a434-954075d71b20 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:48.672626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 503/995] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Jul 12 21:53:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:48.673936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e7e37-13b5-43e7-a434-954075d71b20 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:48.686339 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:48.735771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195c6bd-d4e6-4c0d-b761-004f6327c7da None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:48.789097 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ece2c8-7fd5-443f-a144-9914d9529a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:48.790612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ece2c8-7fd5-443f-a144-9914d9529a53 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:48.790612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ece2c8-7fd5-443f-a144-9914d9529a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:48.790612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ece2c8-7fd5-443f-a144-9914d9529a53 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:48.791387 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ece2c8-7fd5-443f-a144-9914d9529a53 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:48.793687 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ece2c8-7fd5-443f-a144-9914d9529a53 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:48.807541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-02ece2c8-7fd5-443f-a144-9914d9529a53 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 21:53:48.807541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:48.807541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:48.807541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:48.807541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:48.807541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:48.807541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:48.807541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:48.807541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 21:53:48.807541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:48.807541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:48.807541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:48.807541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:48.807541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:48.807541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:48.807541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:48.807541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:48.809668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:48.809668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:48.809668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:48.809668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:48.809668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:48.809668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:48.809668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:48.809668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:48.809668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:48.809668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:48.809668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:48.809668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:48.809668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:48.809668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:48.809668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:48.809668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:48.811366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:48.811366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:48.811366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:48.811366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:48.811366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:48.811366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:48.811366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:48.811366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:48.811366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:48.811366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:48.811366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:48.811366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:48.811366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 21:53:48.811366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:48.811366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 493/996] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:48.812755 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e7e37-13b5-43e7-a434-954075d71b20 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:48.814833 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ece2c8-7fd5-443f-a144-9914d9529a53 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:48.851412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfd029-0e8d-4f06-87ae-486953f38f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:48.852092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfd029-0e8d-4f06-87ae-486953f38f12 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:48.852375 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfd029-0e8d-4f06-87ae-486953f38f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:48.852779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfd029-0e8d-4f06-87ae-486953f38f12 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:48.853482 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfd029-0e8d-4f06-87ae-486953f38f12 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:48.853788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfd029-0e8d-4f06-87ae-486953f38f12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:48.855269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfd029-0e8d-4f06-87ae-486953f38f12 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:48.865346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 504/997] 10.222.0.44 () {56 vars in 1122 bytes} [Tue Jul 12 21:53:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:48.866808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfd029-0e8d-4f06-87ae-486953f38f12 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:48.929699 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:48.935792 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ece2c8-7fd5-443f-a144-9914d9529a53 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:48.940946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:48.976622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfd029-0e8d-4f06-87ae-486953f38f12 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:48.981613 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805c8c4-7c37-4aef-905a-2c02cb8aa09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:48.982503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805c8c4-7c37-4aef-905a-2c02cb8aa09a None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:48.982996 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805c8c4-7c37-4aef-905a-2c02cb8aa09a None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:48.983474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805c8c4-7c37-4aef-905a-2c02cb8aa09a None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:48.984368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805c8c4-7c37-4aef-905a-2c02cb8aa09a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:48.984892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805c8c4-7c37-4aef-905a-2c02cb8aa09a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:48.986462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805c8c4-7c37-4aef-905a-2c02cb8aa09a None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:48.993300 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 494/998] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Jul 12 21:53:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:48.998434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805c8c4-7c37-4aef-905a-2c02cb8aa09a None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:49.016796 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b188da-c098-4d83-a5d8-04da7f9e4bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:49.017541 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b188da-c098-4d83-a5d8-04da7f9e4bbf None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:49.017842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b188da-c098-4d83-a5d8-04da7f9e4bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:49.018094 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b188da-c098-4d83-a5d8-04da7f9e4bbf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:49.018835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b188da-c098-4d83-a5d8-04da7f9e4bbf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:49.020200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b188da-c098-4d83-a5d8-04da7f9e4bbf None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:49.044088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 505/999] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Jul 12 21:53:48 2022] POST /identity/v3/endpoints => generated 325 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:53:49.044088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed9500-c307-4169-aab5-f8fb87615415 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:49.044088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed9500-c307-4169-aab5-f8fb87615415 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:49.044088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed9500-c307-4169-aab5-f8fb87615415 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:49.044088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 506/1000] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:49.045998 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020329fa-0d82-45ee-a826-a7f2434fb860 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:49.046292 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020329fa-0d82-45ee-a826-a7f2434fb860 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:49.046574 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020329fa-0d82-45ee-a826-a7f2434fb860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:49.064690 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-020329fa-0d82-45ee-a826-a7f2434fb860 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:49.065949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-020329fa-0d82-45ee-a826-a7f2434fb860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:49.121508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805c8c4-7c37-4aef-905a-2c02cb8aa09a None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:49.168747 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 507/1001] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:49 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:53:49.173823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fab3f-5fec-4d9e-8693-b77aa7f9f042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:49.174503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fab3f-5fec-4d9e-8693-b77aa7f9f042 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:49.174784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fab3f-5fec-4d9e-8693-b77aa7f9f042 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:49.175105 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fab3f-5fec-4d9e-8693-b77aa7f9f042 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:49.175816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06fab3f-5fec-4d9e-8693-b77aa7f9f042 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:49.176121 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06fab3f-5fec-4d9e-8693-b77aa7f9f042 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:49.177525 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06fab3f-5fec-4d9e-8693-b77aa7f9f042 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:49.188871 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 495/1002] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Jul 12 21:53:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:49.190127 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020329fa-0d82-45ee-a826-a7f2434fb860 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:49.194156 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fab3f-5fec-4d9e-8693-b77aa7f9f042 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:49.223760 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:49.260235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:49.307858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020329fa-0d82-45ee-a826-a7f2434fb860 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:49.314545 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fab3f-5fec-4d9e-8693-b77aa7f9f042 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:49.343429 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e953c-1032-4757-b379-80d948415ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:49.344097 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e953c-1032-4757-b379-80d948415ebe None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:49.344389 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e953c-1032-4757-b379-80d948415ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:49.344686 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e953c-1032-4757-b379-80d948415ebe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:49.345490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e953c-1032-4757-b379-80d948415ebe None admin] RBAC: Adding query filter params (interface=public, service_id=3b3304eab18b4386be3fc8d3d1c39854, region_id=RegionOne) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:49.345808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e953c-1032-4757-b379-80d948415ebe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:49.347189 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e953c-1032-4757-b379-80d948415ebe None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:49.352864 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 508/1003] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Jul 12 21:53:49 2022] GET /identity/v3/endpoints?service_id=3b3304eab18b4386be3fc8d3d1c39854&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:49.353185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601578f-961a-4aa7-8a16-d973210fecbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:49.353984 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601578f-961a-4aa7-8a16-d973210fecbb None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:49.354271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601578f-961a-4aa7-8a16-d973210fecbb None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:49.354580 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601578f-961a-4aa7-8a16-d973210fecbb None admin] PATH_INFO: `/v3/services/image` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:49.355235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d601578f-961a-4aa7-8a16-d973210fecbb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:49.356615 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d601578f-961a-4aa7-8a16-d973210fecbb None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:49.358169 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e953c-1032-4757-b379-80d948415ebe None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:49.369244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-d601578f-961a-4aa7-8a16-d973210fecbb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 21:53:49.369244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:49.369244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:49.369244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:49.369244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:49.369244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:49.369244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:49.369244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:49.369244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 21:53:49.369244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:49.369244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:49.369244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:49.369244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:49.369244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:49.369244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:49.369244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:49.369244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:49.370323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:49.370323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:49.370323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:49.370323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:49.370323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:49.370323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:49.370323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:49.370323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:49.370323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:49.370323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:49.370323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:49.370323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:49.370323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:49.370323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:49.370323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:49.370323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:49.371243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:49.371243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:49.371243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:49.371243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:49.371243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:49.371243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:49.371243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:49.371243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:49.371243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:49.371243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:49.371243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:49.371243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:49.371243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 21:53:49.371243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:49.371243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 496/1004] 10.222.0.44 () {56 vars in 1105 bytes} [Tue Jul 12 21:53:49 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:49.376506 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601578f-961a-4aa7-8a16-d973210fecbb None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:49.468137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e953c-1032-4757-b379-80d948415ebe None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:49.469942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:49.486462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601578f-961a-4aa7-8a16-d973210fecbb None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:49.503214 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:49.516556 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42887b-8544-47f0-ab27-7e50ff6ce56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:49.517255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42887b-8544-47f0-ab27-7e50ff6ce56f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:49.517571 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42887b-8544-47f0-ab27-7e50ff6ce56f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:49.517854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42887b-8544-47f0-ab27-7e50ff6ce56f None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:49.518770 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d42887b-8544-47f0-ab27-7e50ff6ce56f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:49.519899 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d42887b-8544-47f0-ab27-7e50ff6ce56f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:49.527250 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 509/1005] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Jul 12 21:53:49 2022] GET /identity/v3/services => generated 1727 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 21:53:49.533032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f216017-55a9-4855-9a18-15c6f48bff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:49.533727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f216017-55a9-4855-9a18-15c6f48bff7b None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:49.534017 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f216017-55a9-4855-9a18-15c6f48bff7b None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:49.534294 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f216017-55a9-4855-9a18-15c6f48bff7b None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:49.535024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f216017-55a9-4855-9a18-15c6f48bff7b None admin] RBAC: Adding query filter params (name=image) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:49.535365 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f216017-55a9-4855-9a18-15c6f48bff7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:49.536867 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f216017-55a9-4855-9a18-15c6f48bff7b None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:49.548063 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 497/1006] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:49.549785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42887b-8544-47f0-ab27-7e50ff6ce56f None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:49.660545 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42887b-8544-47f0-ab27-7e50ff6ce56f None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:49.698276 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d34ec-b6a5-4bdd-ba43-613d7b746cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:49.698958 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d34ec-b6a5-4bdd-ba43-613d7b746cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:49.699243 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d34ec-b6a5-4bdd-ba43-613d7b746cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:49.699524 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d34ec-b6a5-4bdd-ba43-613d7b746cf4 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:49.700250 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d34ec-b6a5-4bdd-ba43-613d7b746cf4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:49.700559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d34ec-b6a5-4bdd-ba43-613d7b746cf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:49.702057 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d34ec-b6a5-4bdd-ba43-613d7b746cf4 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:49.707657 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 510/1007] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:49 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:51.122144 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa019056-7c54-4f0c-946e-b35ad2b0cfab None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:51.122696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa019056-7c54-4f0c-946e-b35ad2b0cfab None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:51.122696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa019056-7c54-4f0c-946e-b35ad2b0cfab None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:51.124741 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 498/1008] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:51.129307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef4313-2bc8-4624-b831-7df848d5cbad None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:51.129835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef4313-2bc8-4624-b831-7df848d5cbad None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:51.130125 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef4313-2bc8-4624-b831-7df848d5cbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:51.148325 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-5aef4313-2bc8-4624-b831-7df848d5cbad None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:51.149856 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5aef4313-2bc8-4624-b831-7df848d5cbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:51.201627 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:51.228821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 511/1009] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:51 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:51.248449 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa019056-7c54-4f0c-946e-b35ad2b0cfab None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:51.357240 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa019056-7c54-4f0c-946e-b35ad2b0cfab None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:51.393029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dd7a5-1a46-4c68-9e0f-2d8b41da99df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:51.393826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd7a5-1a46-4c68-9e0f-2d8b41da99df None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:51.394112 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd7a5-1a46-4c68-9e0f-2d8b41da99df None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:51.394392 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd7a5-1a46-4c68-9e0f-2d8b41da99df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:51.395579 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dd7a5-1a46-4c68-9e0f-2d8b41da99df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:51.397133 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dd7a5-1a46-4c68-9e0f-2d8b41da99df None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:51.398764 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 499/1010] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:51.404892 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef4313-2bc8-4624-b831-7df848d5cbad None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:51.508044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef4313-2bc8-4624-b831-7df848d5cbad None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:51.543026 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec67be-bfe9-4eb0-b7fb-42c737332fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:51.543698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec67be-bfe9-4eb0-b7fb-42c737332fa2 None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:51.543982 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec67be-bfe9-4eb0-b7fb-42c737332fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:51.544269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec67be-bfe9-4eb0-b7fb-42c737332fa2 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:51.545014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec67be-bfe9-4eb0-b7fb-42c737332fa2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:51.546463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec67be-bfe9-4eb0-b7fb-42c737332fa2 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:51.580742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 512/1011] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Jul 12 21:53:51 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:53:51.687825 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc0be4-aebf-4fe2-ad6d-71f232900c2e None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:51.688157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc0be4-aebf-4fe2-ad6d-71f232900c2e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:51.688386 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc0be4-aebf-4fe2-ad6d-71f232900c2e None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:51.689990 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 500/1012] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:51.695846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e76996-486a-4580-b9d7-b326de1eaceb None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:51.696162 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e76996-486a-4580-b9d7-b326de1eaceb None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:51.696407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e76996-486a-4580-b9d7-b326de1eaceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:51.714214 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-a9e76996-486a-4580-b9d7-b326de1eaceb None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:51.715444 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a9e76996-486a-4580-b9d7-b326de1eaceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:51.748567 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:51.769066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55429bdd-f3fa-46c5-af48-635f2e454012 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:51.769352 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55429bdd-f3fa-46c5-af48-635f2e454012 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:51.769636 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55429bdd-f3fa-46c5-af48-635f2e454012 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:51.771002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 501/1013] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:51.777409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4025f4-e916-4211-9979-fca2be309f48 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:51.777707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4025f4-e916-4211-9979-fca2be309f48 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:51.777991 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4025f4-e916-4211-9979-fca2be309f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:51.797018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 513/1014] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:51 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:51.798934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-fd4025f4-e916-4211-9979-fca2be309f48 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:51.800155 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-fd4025f4-e916-4211-9979-fca2be309f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:51.815705 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:51.824703 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e76996-486a-4580-b9d7-b326de1eaceb None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:51.884763 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 502/1015] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:51 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:53:51.907640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4025f4-e916-4211-9979-fca2be309f48 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:51.939068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e76996-486a-4580-b9d7-b326de1eaceb None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:51.975633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6099ee-e5c3-4a5c-b0c8-460e6cfcdb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:51.976401 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6099ee-e5c3-4a5c-b0c8-460e6cfcdb0c None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:51.976683 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6099ee-e5c3-4a5c-b0c8-460e6cfcdb0c None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:51.976987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6099ee-e5c3-4a5c-b0c8-460e6cfcdb0c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:51.978038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6099ee-e5c3-4a5c-b0c8-460e6cfcdb0c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:51.979016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6099ee-e5c3-4a5c-b0c8-460e6cfcdb0c None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:51.990859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-3f6099ee-e5c3-4a5c-b0c8-460e6cfcdb0c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 21:53:51.990859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:51.990859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:51.990859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:51.990859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:51.990859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:51.990859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:51.990859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:51.990859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 21:53:51.990859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:51.990859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:51.990859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:51.990859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:51.990859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:51.990859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:51.990859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:51.990859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:51.992076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:51.992076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:51.992076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:51.992076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:51.992076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:51.992076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:51.992076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:51.992076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:51.992076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:51.992076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:51.992076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:51.992076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:51.992076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:51.992076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:51.992076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:51.992076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:51.992990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:51.992990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:51.992990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:51.992990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:51.992990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:51.992990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:51.992990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:51.992990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:51.992990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:51.992990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:51.992990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:51.992990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:51.992990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 21:53:51.992990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:51.992990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 514/1016] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:52.001994 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6099ee-e5c3-4a5c-b0c8-460e6cfcdb0c None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:52.032813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:52.033795 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4025f4-e916-4211-9979-fca2be309f48 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:52.070637 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4a27c-88a9-4ac5-b6e6-d6fc183d77bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:52.071312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4a27c-88a9-4ac5-b6e6-d6fc183d77bf None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:52.071597 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4a27c-88a9-4ac5-b6e6-d6fc183d77bf None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:52.071881 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4a27c-88a9-4ac5-b6e6-d6fc183d77bf None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:52.072606 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f4a27c-88a9-4ac5-b6e6-d6fc183d77bf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:52.074015 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f4a27c-88a9-4ac5-b6e6-d6fc183d77bf None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:52.089112 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 503/1017] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Jul 12 21:53:51 2022] POST /identity/v3/services => generated 242 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 21:53:52.119980 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6099ee-e5c3-4a5c-b0c8-460e6cfcdb0c None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:52.168196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81131438-9f58-4de1-a4ab-db326675b76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:52.169111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81131438-9f58-4de1-a4ab-db326675b76c None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:52.169111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81131438-9f58-4de1-a4ab-db326675b76c None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:52.169450 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81131438-9f58-4de1-a4ab-db326675b76c None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:52.170096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81131438-9f58-4de1-a4ab-db326675b76c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:52.170480 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81131438-9f58-4de1-a4ab-db326675b76c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:52.171913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81131438-9f58-4de1-a4ab-db326675b76c None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:52.178082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 515/1018] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Jul 12 21:53:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:52.183860 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4a27c-88a9-4ac5-b6e6-d6fc183d77bf None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:52.198164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:52.307096 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4a27c-88a9-4ac5-b6e6-d6fc183d77bf None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:52.342054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8473c-328e-4a5e-a81a-7c08aed3d7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:52.342782 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8473c-328e-4a5e-a81a-7c08aed3d7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:52.343069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8473c-328e-4a5e-a81a-7c08aed3d7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:52.343347 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8473c-328e-4a5e-a81a-7c08aed3d7e5 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:52.344113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8473c-328e-4a5e-a81a-7c08aed3d7e5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:52.344428 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8473c-328e-4a5e-a81a-7c08aed3d7e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:52.345850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8473c-328e-4a5e-a81a-7c08aed3d7e5 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:52.356775 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 504/1019] 10.222.0.44 () {56 vars in 1112 bytes} [Tue Jul 12 21:53:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:52.358008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81131438-9f58-4de1-a4ab-db326675b76c None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:52.447567 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:52.463848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81131438-9f58-4de1-a4ab-db326675b76c None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:52.498757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b26dcc-1c66-45c8-9379-1335d66e445d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:52.499418 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b26dcc-1c66-45c8-9379-1335d66e445d None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:52.499707 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b26dcc-1c66-45c8-9379-1335d66e445d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:52.499987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b26dcc-1c66-45c8-9379-1335d66e445d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:52.500765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b26dcc-1c66-45c8-9379-1335d66e445d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:52.502139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b26dcc-1c66-45c8-9379-1335d66e445d None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:52.522319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 516/1020] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Jul 12 21:53:52 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 21:53:53.718659 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9546e06-2259-4abb-bbc3-3684f2694f42 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:53.719254 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9546e06-2259-4abb-bbc3-3684f2694f42 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:53.719254 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9546e06-2259-4abb-bbc3-3684f2694f42 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:53.720863 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 505/1021] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:53.727027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e153e-45be-4adc-980e-0d306a28ae95 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:53.727589 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e153e-45be-4adc-980e-0d306a28ae95 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:53.727589 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e153e-45be-4adc-980e-0d306a28ae95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:53.745302 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-d48e153e-45be-4adc-980e-0d306a28ae95 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:53.746511 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d48e153e-45be-4adc-980e-0d306a28ae95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:53.825348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 517/1022] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:53.851161 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9546e06-2259-4abb-bbc3-3684f2694f42 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:53.927377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:53.969338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9546e06-2259-4abb-bbc3-3684f2694f42 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:54.005229 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de51e4-3502-4892-80d4-c08e58249387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:54.006093 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de51e4-3502-4892-80d4-c08e58249387 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:54.006376 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de51e4-3502-4892-80d4-c08e58249387 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:54.006655 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de51e4-3502-4892-80d4-c08e58249387 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:54.007847 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de51e4-3502-4892-80d4-c08e58249387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:54.009337 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de51e4-3502-4892-80d4-c08e58249387 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:54.015503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 506/1023] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:54.017352 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e153e-45be-4adc-980e-0d306a28ae95 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:54.087460 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:54.124549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e153e-45be-4adc-980e-0d306a28ae95 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:54.157570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585f85c-5f98-4440-8303-d12591f48292 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:54.158283 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585f85c-5f98-4440-8303-d12591f48292 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:54.158794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585f85c-5f98-4440-8303-d12591f48292 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:54.159552 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8d329-48db-4e36-89ba-e533d2fe1118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:54.160230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8d329-48db-4e36-89ba-e533d2fe1118 None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:54.160513 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8d329-48db-4e36-89ba-e533d2fe1118 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:54.160823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8d329-48db-4e36-89ba-e533d2fe1118 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:54.161571 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8d329-48db-4e36-89ba-e533d2fe1118 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:54.161784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 507/1024] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:54.163937 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8d329-48db-4e36-89ba-e533d2fe1118 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:54.167018 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09931fbb-63b9-4d31-917b-d4ba0fdfcebf None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:54.167313 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09931fbb-63b9-4d31-917b-d4ba0fdfcebf None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:54.167594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09931fbb-63b9-4d31-917b-d4ba0fdfcebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:54.186418 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 518/1025] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:53:54 2022] POST /identity/v3/users => generated 285 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 21:53:54.186629 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-09931fbb-63b9-4d31-917b-d4ba0fdfcebf None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:54.187503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-09931fbb-63b9-4d31-917b-d4ba0fdfcebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:54.265803 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 508/1026] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:53:54.289299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8d329-48db-4e36-89ba-e533d2fe1118 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:54.406745 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8d329-48db-4e36-89ba-e533d2fe1118 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:54.432765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:54.446733 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9e847-8a74-4569-993b-948e9b065fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:54.447530 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9e847-8a74-4569-993b-948e9b065fbd None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:54.447819 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9e847-8a74-4569-993b-948e9b065fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:54.448102 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9e847-8a74-4569-993b-948e9b065fbd None admin] PATH_INFO: `/v3/services/image` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:54.448855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9e847-8a74-4569-993b-948e9b065fbd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:54.450245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9e847-8a74-4569-993b-948e9b065fbd None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:54.462854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-aed9e847-8a74-4569-993b-948e9b065fbd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 21:53:54.462854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:54.462854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:54.462854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:54.462854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:54.462854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:54.462854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:54.462854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:54.462854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 21:53:54.462854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:54.462854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:54.462854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:54.462854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:54.462854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:54.462854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:54.462854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:54.462854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:54.463874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:54.463874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:54.463874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:54.463874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:54.463874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:54.463874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:54.463874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:54.463874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:54.463874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:54.463874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:54.463874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:54.463874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:54.463874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:54.463874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:54.463874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:54.463874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:54.464762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:54.464762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:54.464762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:54.464762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:54.464762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:54.464762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:54.464762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:54.464762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:54.464762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:54.464762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:54.464762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:54.464762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:54.464762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 21:53:54.464762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:54.464762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 519/1027] 10.222.0.44 () {56 vars in 1105 bytes} [Tue Jul 12 21:53:54 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:54.470213 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09931fbb-63b9-4d31-917b-d4ba0fdfcebf None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:54.542261 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:54.576208 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09931fbb-63b9-4d31-917b-d4ba0fdfcebf None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:54.615215 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7ff88-4116-47ff-922f-0ba69bc86fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:54.615892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7ff88-4116-47ff-922f-0ba69bc86fee None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:54.616253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7ff88-4116-47ff-922f-0ba69bc86fee None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:54.616454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7ff88-4116-47ff-922f-0ba69bc86fee None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:54.617196 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7ff88-4116-47ff-922f-0ba69bc86fee None admin] RBAC: Adding query filter params (name=image) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:54.617493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7ff88-4116-47ff-922f-0ba69bc86fee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:54.618878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7ff88-4116-47ff-922f-0ba69bc86fee None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:54.624197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 509/1028] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:54.629815 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9e847-8a74-4569-993b-948e9b065fbd None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:54.676860 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3a559-2a6f-4e7e-bb83-fda9f4bc5877 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:54.677205 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3a559-2a6f-4e7e-bb83-fda9f4bc5877 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:54.677430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3a559-2a6f-4e7e-bb83-fda9f4bc5877 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:54.679144 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 510/1029] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:53:54.683873 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081b31b-3239-48f3-9c24-58472e330cc0 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:54.684163 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081b31b-3239-48f3-9c24-58472e330cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:54.684444 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081b31b-3239-48f3-9c24-58472e330cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:54.702031 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-7081b31b-3239-48f3-9c24-58472e330cc0 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:54.703216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-7081b31b-3239-48f3-9c24-58472e330cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:54.748472 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9e847-8a74-4569-993b-948e9b065fbd None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:54.781956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 511/1030] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:53:54.789115 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ee028-7c98-4e33-b5ab-94f97ff0a3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:54.789796 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ee028-7c98-4e33-b5ab-94f97ff0a3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:54.790079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ee028-7c98-4e33-b5ab-94f97ff0a3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:54.790361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ee028-7c98-4e33-b5ab-94f97ff0a3c8 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:54.791073 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ee028-7c98-4e33-b5ab-94f97ff0a3c8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:54.791386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ee028-7c98-4e33-b5ab-94f97ff0a3c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:54.792845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ee028-7c98-4e33-b5ab-94f97ff0a3c8 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:54.800755 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 520/1031] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:54.803816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081b31b-3239-48f3-9c24-58472e330cc0 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:54.806205 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ee028-7c98-4e33-b5ab-94f97ff0a3c8 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:54.867031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:54.874168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:54.923620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081b31b-3239-48f3-9c24-58472e330cc0 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:54.931022 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ee028-7c98-4e33-b5ab-94f97ff0a3c8 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:54.965354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82179573-d157-416c-a94a-58fe8eeeffe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:54.966253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82179573-d157-416c-a94a-58fe8eeeffe8 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:54.966334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82179573-d157-416c-a94a-58fe8eeeffe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:54.966879 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82179573-d157-416c-a94a-58fe8eeeffe8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:54.967979 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82179573-d157-416c-a94a-58fe8eeeffe8 None admin] RBAC: Adding query filter params (interface=public, service_id=fae680d7f78146e0a45f2fe02a45fdee, region_id=RegionOne) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:54.968584 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82179573-d157-416c-a94a-58fe8eeeffe8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:54.970096 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82179573-d157-416c-a94a-58fe8eeeffe8 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:54.971629 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab6c28-7df9-4ae8-99c2-30e4355dffe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:54.972440 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab6c28-7df9-4ae8-99c2-30e4355dffe3 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:54.972994 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab6c28-7df9-4ae8-99c2-30e4355dffe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:54.973126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab6c28-7df9-4ae8-99c2-30e4355dffe3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:54.977986 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 512/1032] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Jul 12 21:53:54 2022] GET /identity/v3/endpoints?service_id=fae680d7f78146e0a45f2fe02a45fdee&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:54.983127 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82179573-d157-416c-a94a-58fe8eeeffe8 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:54.984968 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab6c28-7df9-4ae8-99c2-30e4355dffe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:54.993426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-d3ab6c28-7df9-4ae8-99c2-30e4355dffe3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 21:53:54.993426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:54.993426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:54.993426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:54.993426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:54.993426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:54.993426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:54.993426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:54.993426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:54.993426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:54.993426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:54.993426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:54.993426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:54.993426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:54.993426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:54.993426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:54.994391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:54.994391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:54.994391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:54.994391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:54.994391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:54.994391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:54.994391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:54.994391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:54.994391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:54.994391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:54.994391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:54.994391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:54.994391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:54.994391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:54.994391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:54.994391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:54.995266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:54.995266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:54.995266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:54.995266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:54.995266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:54.995266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:54.995266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:54.995266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:54.995266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:54.995266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:54.995266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 21:53:54.995266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:54.995266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 521/1033] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Jul 12 21:53:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:55.000353 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab6c28-7df9-4ae8-99c2-30e4355dffe3 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:55.099684 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82179573-d157-416c-a94a-58fe8eeeffe8 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:55.109038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab6c28-7df9-4ae8-99c2-30e4355dffe3 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:55.122897 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:55.129666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:55.139785 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7293f75-e236-4609-bb25-226a1523f3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:55.140430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7293f75-e236-4609-bb25-226a1523f3ee None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:55.140735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7293f75-e236-4609-bb25-226a1523f3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:55.141008 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7293f75-e236-4609-bb25-226a1523f3ee None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:55.141687 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7293f75-e236-4609-bb25-226a1523f3ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:55.143052 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7293f75-e236-4609-bb25-226a1523f3ee None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:55.147973 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00992cfa-c11d-45bd-a945-ed8fbe09ccf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:55.148737 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00992cfa-c11d-45bd-a945-ed8fbe09ccf5 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:55.149074 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00992cfa-c11d-45bd-a945-ed8fbe09ccf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:55.149357 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00992cfa-c11d-45bd-a945-ed8fbe09ccf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:55.150107 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00992cfa-c11d-45bd-a945-ed8fbe09ccf5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:55.150419 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00992cfa-c11d-45bd-a945-ed8fbe09ccf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:55.160749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 513/1034] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Jul 12 21:53:54 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 21:53:55.160981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00992cfa-c11d-45bd-a945-ed8fbe09ccf5 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:55.169794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 522/1035] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Jul 12 21:53:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:55.175621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7293f75-e236-4609-bb25-226a1523f3ee None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:55.290792 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7293f75-e236-4609-bb25-226a1523f3ee None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:55.327931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489b226-7992-4754-9fd5-882fa6c6b93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:55.328768 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489b226-7992-4754-9fd5-882fa6c6b93c None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:55.329057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489b226-7992-4754-9fd5-882fa6c6b93c None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:55.329339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489b226-7992-4754-9fd5-882fa6c6b93c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:55.341535 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489b226-7992-4754-9fd5-882fa6c6b93c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:55.343002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489b226-7992-4754-9fd5-882fa6c6b93c None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:55.360672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-1489b226-7992-4754-9fd5-882fa6c6b93c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:55.360672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:55.360672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:55.360672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:55.360672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:55.360672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:55.360672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:55.360672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:55.360672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:55.360672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:55.360672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:55.360672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:55.360672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:55.360672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:55.360672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:55.361808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:55.361808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:55.361808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:55.361808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:55.361808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:55.361808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:55.361808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:55.361808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:55.361808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:55.361808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:55.361808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:55.361808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:55.361808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:55.361808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:55.361808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:55.361808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:55.361808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:55.362922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:55.362922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:55.362922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:55.362922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:55.362922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 514/1036] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:55.369107 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00992cfa-c11d-45bd-a945-ed8fbe09ccf5 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:55.478990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00992cfa-c11d-45bd-a945-ed8fbe09ccf5 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:55.514932 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1969d0d-a8a1-4896-b7f1-f7109a379f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:55.515899 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1969d0d-a8a1-4896-b7f1-f7109a379f6f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:55.516390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1969d0d-a8a1-4896-b7f1-f7109a379f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:55.516878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1969d0d-a8a1-4896-b7f1-f7109a379f6f None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:55.517789 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1969d0d-a8a1-4896-b7f1-f7109a379f6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:55.518351 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1969d0d-a8a1-4896-b7f1-f7109a379f6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:55.519916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1969d0d-a8a1-4896-b7f1-f7109a379f6f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:55.535088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 523/1037] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:55.541221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489b226-7992-4754-9fd5-882fa6c6b93c None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:55.589735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:55.649387 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489b226-7992-4754-9fd5-882fa6c6b93c None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:55.685023 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f5c43-cd49-49d2-ad47-1a49877860cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:55.685831 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f5c43-cd49-49d2-ad47-1a49877860cd None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:55.686126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f5c43-cd49-49d2-ad47-1a49877860cd None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:55.686406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f5c43-cd49-49d2-ad47-1a49877860cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:55.695410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f5c43-cd49-49d2-ad47-1a49877860cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:55.696896 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f5c43-cd49-49d2-ad47-1a49877860cd None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:55.711146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-fc6f5c43-cd49-49d2-ad47-1a49877860cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:53:55.711146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:55.711146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:55.711146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:55.711146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:55.711146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:55.711146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:55.711146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:55.711146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:55.711146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:55.711146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:55.711146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:55.711146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:53:55.711146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:53:55.711146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:53:55.711146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:55.712137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:55.718499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:55.718499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:55.718499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:53:55.718499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:53:55.718499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:53:55.718499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:53:55.718499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:53:55.718499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:53:55.718499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:53:55.718499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:55.718499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 515/1038] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Jul 12 21:53:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:55.720540 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1969d0d-a8a1-4896-b7f1-f7109a379f6f None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:55.758204 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:55.827950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1969d0d-a8a1-4896-b7f1-f7109a379f6f None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:55.863289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0c1ea-2b60-4f6b-9e74-31139e064547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:55.863960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0c1ea-2b60-4f6b-9e74-31139e064547 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:55.864246 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0c1ea-2b60-4f6b-9e74-31139e064547 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:55.864525 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0c1ea-2b60-4f6b-9e74-31139e064547 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:55.865936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0c1ea-2b60-4f6b-9e74-31139e064547 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:55.866249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0c1ea-2b60-4f6b-9e74-31139e064547 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:55.867691 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0c1ea-2b60-4f6b-9e74-31139e064547 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:55.892481 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 524/1039] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Jul 12 21:53:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:55.894267 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f5c43-cd49-49d2-ad47-1a49877860cd None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:56.008703 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f5c43-cd49-49d2-ad47-1a49877860cd None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:56.011870 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:56.049910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58661f-7ab1-46af-aa1b-f2984c834b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:56.050576 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58661f-7ab1-46af-aa1b-f2984c834b58 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:56.050859 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58661f-7ab1-46af-aa1b-f2984c834b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:56.051141 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58661f-7ab1-46af-aa1b-f2984c834b58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:56.121070 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58661f-7ab1-46af-aa1b-f2984c834b58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:56.122542 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58661f-7ab1-46af-aa1b-f2984c834b58 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:56.230726 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 516/1040] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:53:55 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:53:56.232195 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0c1ea-2b60-4f6b-9e74-31139e064547 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:56.331088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:56.341986 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0c1ea-2b60-4f6b-9e74-31139e064547 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:56.345931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2947fde-0249-4298-9eaf-7360ab591dfc None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:56.346223 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2947fde-0249-4298-9eaf-7360ab591dfc None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:56.346504 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2947fde-0249-4298-9eaf-7360ab591dfc None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:56.348076 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 517/1041] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:56.353889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee77450-58cf-47ea-94e2-ccbe04d34be2 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:56.354191 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee77450-58cf-47ea-94e2-ccbe04d34be2 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:56.354473 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee77450-58cf-47ea-94e2-ccbe04d34be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:56.360372 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:56.374484 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-0ee77450-58cf-47ea-94e2-ccbe04d34be2 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:56.375808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0ee77450-58cf-47ea-94e2-ccbe04d34be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:56.383821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdaceb-ec64-4397-a914-631240b1129a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:56.384626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdaceb-ec64-4397-a914-631240b1129a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:56.384934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdaceb-ec64-4397-a914-631240b1129a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:56.385268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdaceb-ec64-4397-a914-631240b1129a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:56.396062 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdaceb-ec64-4397-a914-631240b1129a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:56.397595 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdaceb-ec64-4397-a914-631240b1129a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:56.417033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-efcdaceb-ec64-4397-a914-631240b1129a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:56.417033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:56.417033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:56.417033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:56.417033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:56.417033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:56.417033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:56.417033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:56.417033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:56.417033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:56.417033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:56.417033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:56.417033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:56.417033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:56.417033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:56.417967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:56.417967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:56.417967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:56.417967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:56.417967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:56.417967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:56.417967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:56.417967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:56.417967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:56.417967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:56.417967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:56.417967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:56.417967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:56.417967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:56.417967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:56.417967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:56.417967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:56.418858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:56.418858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:56.418858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:56.418858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:56.422974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 525/1042] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:56.424008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdaceb-ec64-4397-a914-631240b1129a None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:56.452747 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 518/1043] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:53:56.473619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee77450-58cf-47ea-94e2-ccbe04d34be2 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:56.532386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdaceb-ec64-4397-a914-631240b1129a None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:56.572479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cdefe2-8d00-477c-bbca-304cd22f1f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:56.573212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdefe2-8d00-477c-bbca-304cd22f1f9e None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:56.573503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdefe2-8d00-477c-bbca-304cd22f1f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:56.573785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdefe2-8d00-477c-bbca-304cd22f1f9e None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:56.574506 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cdefe2-8d00-477c-bbca-304cd22f1f9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:56.574822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cdefe2-8d00-477c-bbca-304cd22f1f9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:56.576199 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cdefe2-8d00-477c-bbca-304cd22f1f9e None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:56.584051 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee77450-58cf-47ea-94e2-ccbe04d34be2 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:56.587515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 526/1044] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:56.593185 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cdefe2-8d00-477c-bbca-304cd22f1f9e None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:56.598049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:56.623077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48444da8-0331-4934-853f-3bb50c8c71be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:56.623872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48444da8-0331-4934-853f-3bb50c8c71be None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:56.624155 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48444da8-0331-4934-853f-3bb50c8c71be None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:56.624431 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48444da8-0331-4934-853f-3bb50c8c71be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:56.626505 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:56.639605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48444da8-0331-4934-853f-3bb50c8c71be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:56.648895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-48444da8-0331-4934-853f-3bb50c8c71be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:53:56.648895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:56.648895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:56.648895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:56.648895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:56.648895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:56.648895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:56.648895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:56.648895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:56.648895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:56.648895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:56.648895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:56.648895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:56.648895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:56.648895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:56.648895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:56.649868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:56.649868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:56.649868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:56.649868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:56.649868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:56.649868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:56.649868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:56.649868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:56.649868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:56.649868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:56.649868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:56.649868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:56.649868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:56.649868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:56.649868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:56.649868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:56.651630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:56.651630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:56.651630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:56.651630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:56.651630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:56.651630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:56.651630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:56.651630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:56.651630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:56.651630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:56.651630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:53:56.651630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:53:56.651630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 519/1045] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:53:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:56.655989 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48444da8-0331-4934-853f-3bb50c8c71be None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:56.709910 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cdefe2-8d00-477c-bbca-304cd22f1f9e None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:56.747792 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887ef62-ed82-4b87-b2b2-196e9295cb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:56.748591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887ef62-ed82-4b87-b2b2-196e9295cb7f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:56.748905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887ef62-ed82-4b87-b2b2-196e9295cb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:56.749188 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887ef62-ed82-4b87-b2b2-196e9295cb7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:56.756501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887ef62-ed82-4b87-b2b2-196e9295cb7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:56.758009 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887ef62-ed82-4b87-b2b2-196e9295cb7f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:56.763576 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48444da8-0331-4934-853f-3bb50c8c71be None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:56.775624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-9887ef62-ed82-4b87-b2b2-196e9295cb7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:56.775624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:56.775624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:56.775624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:56.775624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:56.775624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:56.775624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:56.775624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:56.775624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:56.775624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:56.775624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:56.775624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:56.775624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:53:56.775624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:53:56.775624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:53:56.776959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:53:56.776959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:56.776959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:56.776959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:56.776959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:56.776959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:56.776959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:56.776959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:56.776959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:56.776959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:56.776959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:56.776959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:56.776959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:56.776959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:56.776959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:56.776959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:56.776959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:56.777880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:53:56.777880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:53:56.777880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:53:56.777880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:53:56.777880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:53:56.777880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:53:56.777880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:53:56.777880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:56.777880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 527/1046] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:53:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:56.782715 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887ef62-ed82-4b87-b2b2-196e9295cb7f None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:56.803585 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df8f07-650e-4421-a1cd-e8441f78523f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:56.804261 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df8f07-650e-4421-a1cd-e8441f78523f None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:56.804546 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df8f07-650e-4421-a1cd-e8441f78523f None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:56.804863 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df8f07-650e-4421-a1cd-e8441f78523f None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:56.805617 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df8f07-650e-4421-a1cd-e8441f78523f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:56.805928 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df8f07-650e-4421-a1cd-e8441f78523f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:56.807316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df8f07-650e-4421-a1cd-e8441f78523f None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:56.816204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 520/1047] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:53:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:56.822311 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df8f07-650e-4421-a1cd-e8441f78523f None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:56.854438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:56.889997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:56.898985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887ef62-ed82-4b87-b2b2-196e9295cb7f None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:56.936664 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b7f35-cbb8-49ab-aaf8-c64ee1676792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:56.937378 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b7f35-cbb8-49ab-aaf8-c64ee1676792 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:56.937664 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b7f35-cbb8-49ab-aaf8-c64ee1676792 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:56.937944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b7f35-cbb8-49ab-aaf8-c64ee1676792 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:56.938778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b7f35-cbb8-49ab-aaf8-c64ee1676792 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:56.939095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b7f35-cbb8-49ab-aaf8-c64ee1676792 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:56.940526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b7f35-cbb8-49ab-aaf8-c64ee1676792 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:56.940648 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df8f07-650e-4421-a1cd-e8441f78523f None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:56.953794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 528/1048] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:53:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:56.964546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b7f35-cbb8-49ab-aaf8-c64ee1676792 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:56.997531 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed35ca-f1b5-4bf8-ac72-277e04448a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:56.998773 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed35ca-f1b5-4bf8-ac72-277e04448a0c None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:56.999059 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed35ca-f1b5-4bf8-ac72-277e04448a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:56.999339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed35ca-f1b5-4bf8-ac72-277e04448a0c None admin] PATH_INFO: `/v3/users/544fe6c9f42a43f382f3267e47dcb23a` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:57.015006 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed35ca-f1b5-4bf8-ac72-277e04448a0c None admin] RBAC: Authorizing `identity:get_user(user_id=544fe6c9f42a43f382f3267e47dcb23a)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:57.016238 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed35ca-f1b5-4bf8-ac72-277e04448a0c None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:57.018094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 521/1049] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:53:56 2022] GET /identity/v3/users/544fe6c9f42a43f382f3267e47dcb23a => generated 285 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:57.024745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed35ca-f1b5-4bf8-ac72-277e04448a0c None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:57.079574 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b7f35-cbb8-49ab-aaf8-c64ee1676792 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:57.104154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:57.115535 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238df4c-ad44-4d3a-bf4b-de297f6175f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:57.116200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238df4c-ad44-4d3a-bf4b-de297f6175f1 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:57.116485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238df4c-ad44-4d3a-bf4b-de297f6175f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:57.116793 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238df4c-ad44-4d3a-bf4b-de297f6175f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:57.117637 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238df4c-ad44-4d3a-bf4b-de297f6175f1 None admin] RBAC: Adding query filter params (role.id=3ee4a924b5464226903b6ad15ad59ebc, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=24915625354f418eb7bc4d5ada85fdb0) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:57.117952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238df4c-ad44-4d3a-bf4b-de297f6175f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:57.119390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238df4c-ad44-4d3a-bf4b-de297f6175f1 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:57.136745 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 529/1050] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:53:56 2022] GET /identity/v3/role_assignments?user.id=24915625354f418eb7bc4d5ada85fdb0&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=3ee4a924b5464226903b6ad15ad59ebc => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:57.145832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed35ca-f1b5-4bf8-ac72-277e04448a0c None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:57.164079 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:57.188450 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84e052-d209-418a-b30a-01f30e6f9cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:57.189012 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84e052-d209-418a-b30a-01f30e6f9cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:57.189300 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84e052-d209-418a-b30a-01f30e6f9cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:57.189721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84e052-d209-418a-b30a-01f30e6f9cf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:57.205015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed928c6-8225-41d9-a428-c0db673c5dc0 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:57.205338 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed928c6-8225-41d9-a428-c0db673c5dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:57.205584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed928c6-8225-41d9-a428-c0db673c5dc0 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:57.207228 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 530/1051] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:53:57.213521 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c661d8-b4e2-4fe8-94e4-9403c526b6a0 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:57.213771 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c661d8-b4e2-4fe8-94e4-9403c526b6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:57.213983 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c661d8-b4e2-4fe8-94e4-9403c526b6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:57.232302 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-a9c661d8-b4e2-4fe8-94e4-9403c526b6a0 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:57.233566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a9c661d8-b4e2-4fe8-94e4-9403c526b6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:57.268060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84e052-d209-418a-b30a-01f30e6f9cf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:57.269582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84e052-d209-418a-b30a-01f30e6f9cf5 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:57.312844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 531/1052] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:53:57.336252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c661d8-b4e2-4fe8-94e4-9403c526b6a0 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:57.381454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 522/1053] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:53:57 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:53:57.387486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84e052-d209-418a-b30a-01f30e6f9cf5 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:57.404203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:57.446548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c661d8-b4e2-4fe8-94e4-9403c526b6a0 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:57.471593 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:57.493448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52552544-32ed-4c38-b0ee-44ab7030f180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:57.494235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52552544-32ed-4c38-b0ee-44ab7030f180 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:57.494512 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52552544-32ed-4c38-b0ee-44ab7030f180 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:57.494801 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52552544-32ed-4c38-b0ee-44ab7030f180 None admin] PATH_INFO: `/v3/services/image` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:57.495438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52552544-32ed-4c38-b0ee-44ab7030f180 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:57.496846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52552544-32ed-4c38-b0ee-44ab7030f180 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:57.497247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84e052-d209-418a-b30a-01f30e6f9cf5 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:57.509253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-52552544-32ed-4c38-b0ee-44ab7030f180 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 21:53:57.509253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:57.509253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:53:57.509253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:53:57.509253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:53:57.509253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:53:57.509253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:53:57.509253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:57.509253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 21:53:57.509253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:57.509253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:53:57.509253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:57.509253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:57.509253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:57.509253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:57.509253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:57.509253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:57.510268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:57.510268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:57.510268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:57.510268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:57.510268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:57.510268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:57.510268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:53:57.510268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:53:57.510268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:53:57.510268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:53:57.510268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:57.510268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:57.510268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:57.510268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:57.510268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:57.510268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:57.511109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:57.511109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:57.511109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:57.511109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:57.511109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:57.511109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:57.511109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:57.511109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:57.511109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:57.511109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:57.511109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:53:57.511109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:53:57.511109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 21:53:57.511109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:57.511109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 532/1054] 10.222.0.44 () {56 vars in 1105 bytes} [Tue Jul 12 21:53:57 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:57.516212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52552544-32ed-4c38-b0ee-44ab7030f180 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:57.545438 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbd295-7e82-463a-89bf-766a7c683cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:57.546369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbd295-7e82-463a-89bf-766a7c683cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:57.546583 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbd295-7e82-463a-89bf-766a7c683cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:57.546859 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbd295-7e82-463a-89bf-766a7c683cf0 None admin] PATH_INFO: `/v3/projects/a61a6dd1841c420aae780d467411681f` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:57.548059 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbd295-7e82-463a-89bf-766a7c683cf0 None admin] RBAC: Authorizing `identity:get_project(project_id=a61a6dd1841c420aae780d467411681f)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:57.549544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbd295-7e82-463a-89bf-766a7c683cf0 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:57.556287 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 523/1055] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:53:57 2022] GET /identity/v3/projects/a61a6dd1841c420aae780d467411681f => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:57.557467 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbd295-7e82-463a-89bf-766a7c683cf0 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:57.635955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52552544-32ed-4c38-b0ee-44ab7030f180 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:57.649167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:57.670388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbd295-7e82-463a-89bf-766a7c683cf0 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:57.674172 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7bbde-5275-459e-ab79-687c172af8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:57.674844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7bbde-5275-459e-ab79-687c172af8be None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:57.675130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7bbde-5275-459e-ab79-687c172af8be None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:57.675411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7bbde-5275-459e-ab79-687c172af8be None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:57.676122 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7bbde-5275-459e-ab79-687c172af8be None admin] RBAC: Adding query filter params (name=image) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:57.676434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7bbde-5275-459e-ab79-687c172af8be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:57.678542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7bbde-5275-459e-ab79-687c172af8be None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:57.690133 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 533/1056] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:53:57.691947 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7bbde-5275-459e-ab79-687c172af8be None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:57.706311 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223039e7-6702-451e-b882-c639af26fe46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:57.706970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223039e7-6702-451e-b882-c639af26fe46 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:57.707255 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223039e7-6702-451e-b882-c639af26fe46 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:57.707536 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223039e7-6702-451e-b882-c639af26fe46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:57.708368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223039e7-6702-451e-b882-c639af26fe46 None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.project.id=a61a6dd1841c420aae780d467411681f, user.id=544fe6c9f42a43f382f3267e47dcb23a) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:57.708684 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223039e7-6702-451e-b882-c639af26fe46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:57.710381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223039e7-6702-451e-b882-c639af26fe46 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:57.717359 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 524/1057] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:53:57 2022] GET /identity/v3/role_assignments?user.id=544fe6c9f42a43f382f3267e47dcb23a&scope.project.id=a61a6dd1841c420aae780d467411681f&role.id=053d1730ed764973abb1490894e07681 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:57.747817 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:57.812212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7bbde-5275-459e-ab79-687c172af8be None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:57.850807 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b6d03-a732-47a7-88b0-c3ea659b508e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:57.851581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b6d03-a732-47a7-88b0-c3ea659b508e None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:57.851886 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b6d03-a732-47a7-88b0-c3ea659b508e None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:57.852149 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b6d03-a732-47a7-88b0-c3ea659b508e None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:57.852919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b6d03-a732-47a7-88b0-c3ea659b508e None admin] RBAC: Adding query filter params (type=image) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:57.853240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b6d03-a732-47a7-88b0-c3ea659b508e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:57.854753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b6d03-a732-47a7-88b0-c3ea659b508e None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:57.860769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 534/1058] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:57.866872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223039e7-6702-451e-b882-c639af26fe46 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:57.987341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223039e7-6702-451e-b882-c639af26fe46 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:58.022420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b89a61-2044-4013-8942-a41850314512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:58.023349 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b89a61-2044-4013-8942-a41850314512 None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:58.023645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b89a61-2044-4013-8942-a41850314512 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:58.023926 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b89a61-2044-4013-8942-a41850314512 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:58.024668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b89a61-2044-4013-8942-a41850314512 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:58.026089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b89a61-2044-4013-8942-a41850314512 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:58.053026 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 525/1059] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Jul 12 21:53:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:53:59.181500 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82167a7-b3e7-4beb-acac-f0f8729a14f6 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:59.182060 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82167a7-b3e7-4beb-acac-f0f8729a14f6 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:59.182060 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82167a7-b3e7-4beb-acac-f0f8729a14f6 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:59.183676 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 535/1060] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:59.189303 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a14171-6522-4461-89ce-b0ebac53afe7 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:59.189821 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a14171-6522-4461-89ce-b0ebac53afe7 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:59.189920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a14171-6522-4461-89ce-b0ebac53afe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:59.195848 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:59.208535 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-98a14171-6522-4461-89ce-b0ebac53afe7 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:59.210950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-98a14171-6522-4461-89ce-b0ebac53afe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:59.288931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 526/1061] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:53:59.308169 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82167a7-b3e7-4beb-acac-f0f8729a14f6 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:59.413491 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82167a7-b3e7-4beb-acac-f0f8729a14f6 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:59.424027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:59.448766 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f20cbf-56fc-4861-b105-76739e98a7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:59.449584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f20cbf-56fc-4861-b105-76739e98a7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:59.449866 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f20cbf-56fc-4861-b105-76739e98a7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:59.450140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f20cbf-56fc-4861-b105-76739e98a7f8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:59.461684 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f20cbf-56fc-4861-b105-76739e98a7f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:53:59.469880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-63f20cbf-56fc-4861-b105-76739e98a7f8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 21:53:59.469880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:59.469880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:59.469880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:59.469880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:59.469880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:59.469880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:59.469880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:59.469880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:59.469880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:59.469880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:59.469880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:59.469880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:53:59.469880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:53:59.469880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:53:59.469880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:53:59.470784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:53:59.470784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:53:59.470784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:53:59.470784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:53:59.470784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:59.470784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:59.470784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:59.470784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:59.470784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:59.470784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:59.470784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:59.470784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:59.470784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:59.470784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:59.470784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:59.470784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:59.477168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:59.477168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:59.477168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:59.477168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:59.477168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:53:59.477168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:53:59.477168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:53:59.477168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:53:59.477168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:53:59.477168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:53:59.477168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 21:53:59.477168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:59.477168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 536/1062] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Jul 12 21:53:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:53:59.477860 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a14171-6522-4461-89ce-b0ebac53afe7 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:59.580185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a14171-6522-4461-89ce-b0ebac53afe7 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:59.608954 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:59.620762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45d467-b7b4-49e7-b764-1945fa6b6d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:59.621761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45d467-b7b4-49e7-b764-1945fa6b6d5e None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:59.622254 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45d467-b7b4-49e7-b764-1945fa6b6d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:59.622707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45d467-b7b4-49e7-b764-1945fa6b6d5e None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:59.623644 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45d467-b7b4-49e7-b764-1945fa6b6d5e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:59.624131 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45d467-b7b4-49e7-b764-1945fa6b6d5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:59.625741 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45d467-b7b4-49e7-b764-1945fa6b6d5e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:59.636478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 527/1063] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Jul 12 21:53:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:53:59.641930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f20cbf-56fc-4861-b105-76739e98a7f8 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:59.747433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f20cbf-56fc-4861-b105-76739e98a7f8 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:59.771094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b392b-4820-41ad-9435-898474c3219f None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:59.771450 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b392b-4820-41ad-9435-898474c3219f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:59.771751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b392b-4820-41ad-9435-898474c3219f None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:59.773297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 528/1064] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:53:59.778338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216da25d-c724-4acf-8757-e505ce4e491e None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:59.778630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216da25d-c724-4acf-8757-e505ce4e491e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:59.778913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216da25d-c724-4acf-8757-e505ce4e491e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:59.784179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b1ad0-2dfe-42ca-be27-d8cc6ae804f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:59.785012 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b1ad0-2dfe-42ca-be27-d8cc6ae804f1 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:59.785297 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b1ad0-2dfe-42ca-be27-d8cc6ae804f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:59.785578 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b1ad0-2dfe-42ca-be27-d8cc6ae804f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:59.796611 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-216da25d-c724-4acf-8757-e505ce4e491e None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:53:59.796942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b1ad0-2dfe-42ca-be27-d8cc6ae804f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:59.797829 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-216da25d-c724-4acf-8757-e505ce4e491e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:53:59.798339 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b1ad0-2dfe-42ca-be27-d8cc6ae804f1 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:53:59.818202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-7e7b1ad0-2dfe-42ca-be27-d8cc6ae804f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:59.818202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:53:59.818202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:53:59.818202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:53:59.818202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:53:59.818202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:53:59.818202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:53:59.818202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:53:59.818202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:53:59.818202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:53:59.818202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:53:59.818202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:53:59.818202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:53:59.818202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:53:59.818202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:53:59.819154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:53:59.819154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:53:59.819154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:53:59.819154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:53:59.819154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:53:59.819154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:53:59.819154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:53:59.819154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:53:59.819154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:53:59.819154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:53:59.819154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:53:59.819154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:53:59.819154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:53:59.819154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:53:59.819154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:53:59.819154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:53:59.819154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:53:59.820058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:53:59.820058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:53:59.820058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:53:59.820058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:53:59.822308 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 537/1065] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:53:59.827692 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b1ad0-2dfe-42ca-be27-d8cc6ae804f1 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:59.853862 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:53:59.875226 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 529/1066] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:53:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:53:59.899011 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216da25d-c724-4acf-8757-e505ce4e491e None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:53:59.938260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b1ad0-2dfe-42ca-be27-d8cc6ae804f1 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:53:59.982167 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad45c7-0bfb-4548-9991-d9165e3ba156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:53:59.982585 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad45c7-0bfb-4548-9991-d9165e3ba156 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:53:59.982880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad45c7-0bfb-4548-9991-d9165e3ba156 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:53:59.983161 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad45c7-0bfb-4548-9991-d9165e3ba156 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:53:59.983895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad45c7-0bfb-4548-9991-d9165e3ba156 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:53:59.984208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad45c7-0bfb-4548-9991-d9165e3ba156 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:53:59.985632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad45c7-0bfb-4548-9991-d9165e3ba156 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:00.005248 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 538/1067] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:54:00.006879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad45c7-0bfb-4548-9991-d9165e3ba156 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:00.025845 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216da25d-c724-4acf-8757-e505ce4e491e None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:00.047259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:00.063354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084b5a5-69e2-44fc-b460-d1d2fd708b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:00.064150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084b5a5-69e2-44fc-b460-d1d2fd708b6b None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:00.064430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084b5a5-69e2-44fc-b460-d1d2fd708b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:00.064728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084b5a5-69e2-44fc-b460-d1d2fd708b6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:00.076400 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084b5a5-69e2-44fc-b460-d1d2fd708b6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:54:00.084751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-9084b5a5-69e2-44fc-b460-d1d2fd708b6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:54:00.084751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:00.084751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:00.084751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:00.084751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:00.084751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:00.084751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:00.084751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:00.084751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:00.084751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:00.084751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:00.084751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:00.084751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:54:00.084751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:54:00.084751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:54:00.084751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:54:00.085692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:54:00.085692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:54:00.085692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:54:00.085692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:54:00.085692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:00.085692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:00.085692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:00.085692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:00.085692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:00.085692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:00.085692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:00.085692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:00.085692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:00.085692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:00.085692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:00.085692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:00.086520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:00.086520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:00.086520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:00.086520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:00.086520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:54:00.086520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:54:00.086520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:54:00.086520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:54:00.086520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:54:00.086520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:54:00.086520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:54:00.086520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:54:00.088607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 530/1068] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:53:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:54:00.094588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084b5a5-69e2-44fc-b460-d1d2fd708b6b None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:00.100529 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:00.120819 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad45c7-0bfb-4548-9991-d9165e3ba156 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:00.158885 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac1a39-21f5-4756-ba5c-95476ce7f4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:00.159711 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac1a39-21f5-4756-ba5c-95476ce7f4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:00.159994 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac1a39-21f5-4756-ba5c-95476ce7f4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:00.160270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac1a39-21f5-4756-ba5c-95476ce7f4d6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:00.168229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac1a39-21f5-4756-ba5c-95476ce7f4d6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:00.169720 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac1a39-21f5-4756-ba5c-95476ce7f4d6 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:00.185612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-7aac1a39-21f5-4756-ba5c-95476ce7f4d6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:54:00.185612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:00.185612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:00.185612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:00.185612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:00.185612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:00.185612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:00.185612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:00.185612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:00.185612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:00.185612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:00.185612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:00.185612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:54:00.185612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:54:00.185612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:54:00.185612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:00.187058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:00.188212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:00.188212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:00.188212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:54:00.188212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:54:00.188212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:54:00.188212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:54:00.188212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:54:00.188212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:54:00.188212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:54:00.188212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:54:00.188212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 539/1069] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Jul 12 21:54:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:54:00.188212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b78676-a5bd-4ed4-b036-ee2e57e2b895 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:00.188923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b78676-a5bd-4ed4-b036-ee2e57e2b895 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:00.188923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b78676-a5bd-4ed4-b036-ee2e57e2b895 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:00.189970 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 540/1070] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:00.192535 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b78676-a5bd-4ed4-b036-ee2e57e2b895 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:00.210045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084b5a5-69e2-44fc-b460-d1d2fd708b6b None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:00.243950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f04c75-b7e4-4dc1-bb25-92f69d6d1986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:00.244622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f04c75-b7e4-4dc1-bb25-92f69d6d1986 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:00.244937 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f04c75-b7e4-4dc1-bb25-92f69d6d1986 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:00.245227 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f04c75-b7e4-4dc1-bb25-92f69d6d1986 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:00.245978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f04c75-b7e4-4dc1-bb25-92f69d6d1986 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:00.246296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f04c75-b7e4-4dc1-bb25-92f69d6d1986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:00.247678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f04c75-b7e4-4dc1-bb25-92f69d6d1986 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:00.256782 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 531/1071] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:54:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:54:00.258094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594cfde-410e-4e50-9d3f-396dfbe5ce44 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:00.258390 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594cfde-410e-4e50-9d3f-396dfbe5ce44 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:00.258672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594cfde-410e-4e50-9d3f-396dfbe5ce44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:00.274982 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-a594cfde-410e-4e50-9d3f-396dfbe5ce44 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:00.276632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a594cfde-410e-4e50-9d3f-396dfbe5ce44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:00.295206 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b78676-a5bd-4ed4-b036-ee2e57e2b895 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:00.323591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:00.340819 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65db5ff-08e7-4ac5-b937-b1e20d4b8540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:00.341500 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65db5ff-08e7-4ac5-b937-b1e20d4b8540 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:00.341778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65db5ff-08e7-4ac5-b937-b1e20d4b8540 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:00.342053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65db5ff-08e7-4ac5-b937-b1e20d4b8540 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:00.343596 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65db5ff-08e7-4ac5-b937-b1e20d4b8540 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:00.344113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65db5ff-08e7-4ac5-b937-b1e20d4b8540 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:00.345718 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65db5ff-08e7-4ac5-b937-b1e20d4b8540 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:00.360166 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 532/1072] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:00.360701 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:00.361059 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594cfde-410e-4e50-9d3f-396dfbe5ce44 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:00.369821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 541/1073] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Jul 12 21:54:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:00.371285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65db5ff-08e7-4ac5-b937-b1e20d4b8540 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:00.464050 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594cfde-410e-4e50-9d3f-396dfbe5ce44 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:00.476784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65db5ff-08e7-4ac5-b937-b1e20d4b8540 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:00.499507 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1cd5a-22cb-41fe-a317-e6fffd014baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:00.500377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1cd5a-22cb-41fe-a317-e6fffd014baf None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:00.500660 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1cd5a-22cb-41fe-a317-e6fffd014baf None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:00.500961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1cd5a-22cb-41fe-a317-e6fffd014baf None admin] PATH_INFO: `/v3/users/544fe6c9f42a43f382f3267e47dcb23a` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:00.502155 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1cd5a-22cb-41fe-a317-e6fffd014baf None admin] RBAC: Authorizing `identity:get_user(user_id=544fe6c9f42a43f382f3267e47dcb23a)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:00.503610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1cd5a-22cb-41fe-a317-e6fffd014baf None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:00.505272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 533/1074] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:54:00 2022] GET /identity/v3/users/544fe6c9f42a43f382f3267e47dcb23a => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:00.506352 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1cd5a-22cb-41fe-a317-e6fffd014baf None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:00.516785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4341df5-fc14-42b9-8625-dc9ae56894da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:00.517591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4341df5-fc14-42b9-8625-dc9ae56894da None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:00.517870 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4341df5-fc14-42b9-8625-dc9ae56894da None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:00.518153 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4341df5-fc14-42b9-8625-dc9ae56894da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:00.528914 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4341df5-fc14-42b9-8625-dc9ae56894da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:00.530390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4341df5-fc14-42b9-8625-dc9ae56894da None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:00.547026 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:00.548997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-d4341df5-fc14-42b9-8625-dc9ae56894da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:54:00.548997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:00.548997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:00.548997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:00.548997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:00.548997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:00.548997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:00.548997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:00.548997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:00.548997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:00.548997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:00.548997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:00.548997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:54:00.548997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:54:00.548997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:54:00.550148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:54:00.550148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:00.550148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:00.550148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:00.550148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:00.550148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:00.550148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:00.550148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:00.550148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:00.550148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:00.550148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:00.550148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:00.550148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:00.550148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:00.550148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:00.550148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:00.550148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:00.551071 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:54:00.551071 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:54:00.551071 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:54:00.551071 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:54:00.551071 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 542/1075] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:54:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:54:00.551841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4341df5-fc14-42b9-8625-dc9ae56894da None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:00.604259 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:00.611473 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1cd5a-22cb-41fe-a317-e6fffd014baf None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:00.646903 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a788f7-e94d-4061-b11e-ccbadf9f8824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:00.647475 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a788f7-e94d-4061-b11e-ccbadf9f8824 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:00.647761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a788f7-e94d-4061-b11e-ccbadf9f8824 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:00.648047 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a788f7-e94d-4061-b11e-ccbadf9f8824 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:00.654693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4341df5-fc14-42b9-8625-dc9ae56894da None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:00.690585 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da1750-1e78-4e72-85a2-c76df911ae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:00.691720 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da1750-1e78-4e72-85a2-c76df911ae63 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:00.692011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da1750-1e78-4e72-85a2-c76df911ae63 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:00.692290 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da1750-1e78-4e72-85a2-c76df911ae63 None admin] PATH_INFO: `/v3/projects/a61a6dd1841c420aae780d467411681f` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:00.694264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da1750-1e78-4e72-85a2-c76df911ae63 None admin] RBAC: Authorizing `identity:get_project(project_id=a61a6dd1841c420aae780d467411681f)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:00.695719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da1750-1e78-4e72-85a2-c76df911ae63 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:00.697699 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 543/1076] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:54:00 2022] GET /identity/v3/projects/a61a6dd1841c420aae780d467411681f => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:00.698295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da1750-1e78-4e72-85a2-c76df911ae63 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:00.728569 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a788f7-e94d-4061-b11e-ccbadf9f8824 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:00.730192 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a788f7-e94d-4061-b11e-ccbadf9f8824 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:00.771256 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:00.798875 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da1750-1e78-4e72-85a2-c76df911ae63 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:00.820814 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:00.836749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 534/1077] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:54:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:54:00.836749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a788f7-e94d-4061-b11e-ccbadf9f8824 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:00.838658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172d4ef-b689-4346-98b5-42f7bbd79c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:00.839331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172d4ef-b689-4346-98b5-42f7bbd79c27 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:00.839612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172d4ef-b689-4346-98b5-42f7bbd79c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:00.839883 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172d4ef-b689-4346-98b5-42f7bbd79c27 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:00.840608 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172d4ef-b689-4346-98b5-42f7bbd79c27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:00.840955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172d4ef-b689-4346-98b5-42f7bbd79c27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:00.842322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172d4ef-b689-4346-98b5-42f7bbd79c27 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:00.854406 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 544/1078] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:00.855212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172d4ef-b689-4346-98b5-42f7bbd79c27 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:00.937963 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a788f7-e94d-4061-b11e-ccbadf9f8824 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:00.965241 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172d4ef-b689-4346-98b5-42f7bbd79c27 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:00.974396 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13916ac3-4695-4b5d-9e81-223edf0a8706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:00.975289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13916ac3-4695-4b5d-9e81-223edf0a8706 None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:00.975836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13916ac3-4695-4b5d-9e81-223edf0a8706 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:00.975836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13916ac3-4695-4b5d-9e81-223edf0a8706 None admin] PATH_INFO: `/v3/projects/a61a6dd1841c420aae780d467411681f/users/544fe6c9f42a43f382f3267e47dcb23a/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:00.977796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13916ac3-4695-4b5d-9e81-223edf0a8706 None admin] RBAC: Authorizing `identity:create_grant(project_id=a61a6dd1841c420aae780d467411681f, user_id=544fe6c9f42a43f382f3267e47dcb23a, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:00.979239 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13916ac3-4695-4b5d-9e81-223edf0a8706 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:00.988140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:00.999392 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 535/1079] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:54:00 2022] PUT /identity/v3/projects/a61a6dd1841c420aae780d467411681f/users/544fe6c9f42a43f382f3267e47dcb23a/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:54:01.000526 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13916ac3-4695-4b5d-9e81-223edf0a8706 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:01.006336 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7237e6b-085e-45a2-99b4-ea1809aa57a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:01.007534 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7237e6b-085e-45a2-99b4-ea1809aa57a6 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:01.007767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7237e6b-085e-45a2-99b4-ea1809aa57a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:01.007915 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7237e6b-085e-45a2-99b4-ea1809aa57a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:01.022615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7237e6b-085e-45a2-99b4-ea1809aa57a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:01.025149 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7237e6b-085e-45a2-99b4-ea1809aa57a6 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:01.044939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-f7237e6b-085e-45a2-99b4-ea1809aa57a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:54:01.044939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:01.044939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:01.044939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:01.044939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:01.044939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:01.044939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:01.044939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:01.044939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:01.044939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:01.044939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:01.044939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:01.044939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:54:01.044939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:54:01.044939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:54:01.045939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:54:01.045939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:01.045939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:01.045939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:01.045939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:01.045939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:01.045939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:01.045939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:01.045939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:01.045939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:01.045939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:01.045939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:01.045939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:01.045939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:01.045939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:01.045939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:01.045939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:01.046858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:54:01.046858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:54:01.046858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:54:01.046858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:54:01.046858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 545/1080] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:54:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:54:01.053329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7237e6b-085e-45a2-99b4-ea1809aa57a6 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:01.099138 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:01.138509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13916ac3-4695-4b5d-9e81-223edf0a8706 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:01.182410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42df3d2-40a5-4512-8eb2-673b879c6a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:01.183205 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42df3d2-40a5-4512-8eb2-673b879c6a6b None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:01.183485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42df3d2-40a5-4512-8eb2-673b879c6a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:01.183764 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42df3d2-40a5-4512-8eb2-673b879c6a6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:01.187805 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7237e6b-085e-45a2-99b4-ea1809aa57a6 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:01.191073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42df3d2-40a5-4512-8eb2-673b879c6a6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:01.192738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42df3d2-40a5-4512-8eb2-673b879c6a6b None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:01.208572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-f42df3d2-40a5-4512-8eb2-673b879c6a6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:54:01.208572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:01.208572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:01.208572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:01.208572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:01.208572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:01.208572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:01.208572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:01.208572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:01.208572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:01.208572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:01.208572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:01.208572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:54:01.208572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:54:01.208572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:54:01.210109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:54:01.210109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:01.210109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:01.210109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:01.210109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:01.210109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:01.210109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:01.210109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:01.210109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:01.210109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:01.210109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:01.210109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:01.210109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:01.210109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:01.210109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:01.210109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:01.210109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:01.211425 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:54:01.211425 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:54:01.211425 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:54:01.211425 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:54:01.211425 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:54:01.211425 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:54:01.211425 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:54:01.211425 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:54:01.211425 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 536/1081] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:54:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:54:01.217900 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42df3d2-40a5-4512-8eb2-673b879c6a6b None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:01.234428 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351c9d3-3c4e-4914-ab95-8406a5ca47bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:01.235557 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351c9d3-3c4e-4914-ab95-8406a5ca47bd None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:01.235846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351c9d3-3c4e-4914-ab95-8406a5ca47bd None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:01.236261 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351c9d3-3c4e-4914-ab95-8406a5ca47bd None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:01.237029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351c9d3-3c4e-4914-ab95-8406a5ca47bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:01.237420 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351c9d3-3c4e-4914-ab95-8406a5ca47bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:01.238719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351c9d3-3c4e-4914-ab95-8406a5ca47bd None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:01.257130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 546/1082] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:54:01.285249 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:01.335031 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42df3d2-40a5-4512-8eb2-673b879c6a6b None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:01.373838 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad89a9-e1b1-44c6-9986-659d58968868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:01.374519 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad89a9-e1b1-44c6-9986-659d58968868 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:01.374802 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad89a9-e1b1-44c6-9986-659d58968868 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:01.375077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad89a9-e1b1-44c6-9986-659d58968868 None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:01.375887 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad89a9-e1b1-44c6-9986-659d58968868 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:01.376198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad89a9-e1b1-44c6-9986-659d58968868 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:01.377701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad89a9-e1b1-44c6-9986-659d58968868 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:01.395838 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 537/1083] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:54:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:54:01.398080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351c9d3-3c4e-4914-ab95-8406a5ca47bd None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:01.497435 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:01.507993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351c9d3-3c4e-4914-ab95-8406a5ca47bd None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:01.544912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f3c41-b18f-4851-bf4e-0ddfa63cf50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:01.545814 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f3c41-b18f-4851-bf4e-0ddfa63cf50d None admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:01.546099 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f3c41-b18f-4851-bf4e-0ddfa63cf50d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:01.546384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f3c41-b18f-4851-bf4e-0ddfa63cf50d None admin] PATH_INFO: `/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/24915625354f418eb7bc4d5ada85fdb0/roles/3ee4a924b5464226903b6ad15ad59ebc` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:01.548619 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f3c41-b18f-4851-bf4e-0ddfa63cf50d None admin] RBAC: Authorizing `identity:create_grant(project_id=459e4743ff3043f69408ca2095be3bd0, user_id=24915625354f418eb7bc4d5ada85fdb0, role_id=3ee4a924b5464226903b6ad15ad59ebc)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:01.550085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f3c41-b18f-4851-bf4e-0ddfa63cf50d None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:01.564444 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 547/1084] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:54:01 2022] PUT /identity/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/24915625354f418eb7bc4d5ada85fdb0/roles/3ee4a924b5464226903b6ad15ad59ebc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:54:03.043796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90cc05-ce76-49b2-b844-2a80225d475e None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:03.044318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90cc05-ce76-49b2-b844-2a80225d475e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:03.044434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90cc05-ce76-49b2-b844-2a80225d475e None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:03.045871 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 538/1085] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:03.051064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dcd57-e948-45de-a07f-90b43af38ab9 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:03.051601 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dcd57-e948-45de-a07f-90b43af38ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:03.051601 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dcd57-e948-45de-a07f-90b43af38ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:03.069770 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-cf3dcd57-e948-45de-a07f-90b43af38ab9 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:03.071309 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-cf3dcd57-e948-45de-a07f-90b43af38ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:03.137238 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 548/1086] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:03.159863 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90cc05-ce76-49b2-b844-2a80225d475e None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:03.261774 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90cc05-ce76-49b2-b844-2a80225d475e None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:03.281625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:03.295954 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5b605-9072-4fb6-a779-2281cf63d8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:03.296775 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5b605-9072-4fb6-a779-2281cf63d8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:03.297056 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5b605-9072-4fb6-a779-2281cf63d8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:03.297326 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5b605-9072-4fb6-a779-2281cf63d8e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:03.307760 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5b605-9072-4fb6-a779-2281cf63d8e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:54:03.315291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-cfd5b605-9072-4fb6-a779-2281cf63d8e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:54:03.315291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:03.315291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:03.315291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:03.315291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:03.315291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:03.315291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:03.315291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:03.315291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:03.315291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:03.315291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:03.315291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:03.315291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:54:03.315291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:54:03.315291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:54:03.315291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:54:03.316284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:54:03.316284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:54:03.316284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:54:03.316284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:54:03.316284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:03.316284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:03.316284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:03.316284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:03.316284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:03.316284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:03.316284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:03.316284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:03.316284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:03.316284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:03.316284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:03.316284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:03.317416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:03.317416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:03.317416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:03.317416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:03.317416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:54:03.317416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:54:03.317416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:54:03.317416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:54:03.317416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:54:03.317416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:54:03.317416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:54:03.317416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:54:03.320885 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 539/1087] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:54:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:54:03.323422 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3dcd57-e948-45de-a07f-90b43af38ab9 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:03.405989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:03.424502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3dcd57-e948-45de-a07f-90b43af38ab9 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:03.458461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf0dbc-d359-4b07-b288-1187c6dfd194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:03.459747 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf0dbc-d359-4b07-b288-1187c6dfd194 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:03.460027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf0dbc-d359-4b07-b288-1187c6dfd194 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:03.460300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf0dbc-d359-4b07-b288-1187c6dfd194 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:03.461077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf0dbc-d359-4b07-b288-1187c6dfd194 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:03.461381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf0dbc-d359-4b07-b288-1187c6dfd194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:03.462744 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf0dbc-d359-4b07-b288-1187c6dfd194 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:03.471067 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 549/1088] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:54:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:54:03.476809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5b605-9072-4fb6-a779-2281cf63d8e2 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:03.577654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5b605-9072-4fb6-a779-2281cf63d8e2 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:03.605111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81219c3-0538-4c30-8a2d-313c865d9aa5 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:03.605407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81219c3-0538-4c30-8a2d-313c865d9aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:03.605692 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81219c3-0538-4c30-8a2d-313c865d9aa5 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:03.607077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 550/1089] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:54:03.611191 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3846917-a1d9-4afe-bb0a-f0acebdb9fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:03.612060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3846917-a1d9-4afe-bb0a-f0acebdb9fae None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:03.612165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e2ac8-4473-40f4-832b-b2ccbe73cdf7 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:03.612343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3846917-a1d9-4afe-bb0a-f0acebdb9fae None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:03.612435 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e2ac8-4473-40f4-832b-b2ccbe73cdf7 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:03.612623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3846917-a1d9-4afe-bb0a-f0acebdb9fae None admin] PATH_INFO: `/v3/users/544fe6c9f42a43f382f3267e47dcb23a` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:03.612742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e2ac8-4473-40f4-832b-b2ccbe73cdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:03.613841 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3846917-a1d9-4afe-bb0a-f0acebdb9fae None admin] RBAC: Authorizing `identity:get_user(user_id=544fe6c9f42a43f382f3267e47dcb23a)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:03.615291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3846917-a1d9-4afe-bb0a-f0acebdb9fae None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:03.616946 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 540/1090] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:54:03 2022] GET /identity/v3/users/544fe6c9f42a43f382f3267e47dcb23a => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:03.622586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3846917-a1d9-4afe-bb0a-f0acebdb9fae None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:03.629552 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-a91e2ac8-4473-40f4-832b-b2ccbe73cdf7 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:03.631342 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a91e2ac8-4473-40f4-832b-b2ccbe73cdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:03.672834 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:03.700701 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 551/1091] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:54:03.724316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3846917-a1d9-4afe-bb0a-f0acebdb9fae None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:03.725536 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e2ac8-4473-40f4-832b-b2ccbe73cdf7 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:03.759737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97563875-618a-4b08-9e2a-7ff8e938c8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:03.760388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97563875-618a-4b08-9e2a-7ff8e938c8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:03.760679 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97563875-618a-4b08-9e2a-7ff8e938c8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:03.761000 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97563875-618a-4b08-9e2a-7ff8e938c8c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:03.799220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:03.825914 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e2ac8-4473-40f4-832b-b2ccbe73cdf7 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:03.829685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97563875-618a-4b08-9e2a-7ff8e938c8c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:03.831093 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97563875-618a-4b08-9e2a-7ff8e938c8c7 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:03.859881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b9e41-9816-4b4f-ba6c-0aab72984292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:03.861004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b9e41-9816-4b4f-ba6c-0aab72984292 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:03.861289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b9e41-9816-4b4f-ba6c-0aab72984292 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:03.861568 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b9e41-9816-4b4f-ba6c-0aab72984292 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:03.872435 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b9e41-9816-4b4f-ba6c-0aab72984292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:54:03.878710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:03.881697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-6f1b9e41-9816-4b4f-ba6c-0aab72984292 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 21:54:03.881697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:03.881697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:03.881697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:03.881697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:03.881697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:03.881697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:03.881697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:03.881697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:03.881697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:03.881697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:03.881697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:03.881697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:54:03.881697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:54:03.881697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:54:03.881697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:54:03.883023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:54:03.883023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:54:03.883023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:54:03.883023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:54:03.883023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:03.883023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:03.883023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:03.883023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:03.883023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:03.883023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:03.883023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:03.883023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:03.883023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:03.883023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:03.883023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:03.883023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:03.883874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:03.883874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:03.883874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:03.883874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:03.883874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:54:03.883874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:54:03.883874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:54:03.883874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:54:03.883874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:54:03.883874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:54:03.883874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 21:54:03.883874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:54:03.883874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 552/1092] 10.222.0.44 () {56 vars in 1120 bytes} [Tue Jul 12 21:54:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:54:03.889658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b9e41-9816-4b4f-ba6c-0aab72984292 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:03.932830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 541/1093] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:54:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 21:54:03.938576 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97563875-618a-4b08-9e2a-7ff8e938c8c7 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:04.007938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b9e41-9816-4b4f-ba6c-0aab72984292 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:04.041225 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e69a90-b109-4f3b-a885-1e04ce34d389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:04.041900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e69a90-b109-4f3b-a885-1e04ce34d389 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:04.042175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e69a90-b109-4f3b-a885-1e04ce34d389 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:04.042452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e69a90-b109-4f3b-a885-1e04ce34d389 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:04.043193 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e69a90-b109-4f3b-a885-1e04ce34d389 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:04.043499 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e69a90-b109-4f3b-a885-1e04ce34d389 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:04.044895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e69a90-b109-4f3b-a885-1e04ce34d389 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:04.046260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:04.047468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97563875-618a-4b08-9e2a-7ff8e938c8c7 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:04.059309 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 553/1094] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Jul 12 21:54:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:04.060910 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e69a90-b109-4f3b-a885-1e04ce34d389 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:04.082214 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e3ef7-d965-430e-b361-225216cdc333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:04.083003 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e3ef7-d965-430e-b361-225216cdc333 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:04.083285 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e3ef7-d965-430e-b361-225216cdc333 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:04.083629 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e3ef7-d965-430e-b361-225216cdc333 None admin] PATH_INFO: `/v3/projects/a61a6dd1841c420aae780d467411681f` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:04.084949 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e3ef7-d965-430e-b361-225216cdc333 None admin] RBAC: Authorizing `identity:get_project(project_id=a61a6dd1841c420aae780d467411681f)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:04.086390 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e3ef7-d965-430e-b361-225216cdc333 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:04.088091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 542/1095] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:54:03 2022] GET /identity/v3/projects/a61a6dd1841c420aae780d467411681f => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:54:04.094034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e3ef7-d965-430e-b361-225216cdc333 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:04.111143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:04.161385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e69a90-b109-4f3b-a885-1e04ce34d389 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:04.192760 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e3ef7-d965-430e-b361-225216cdc333 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:04.195979 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643fc76-757b-4c9e-8ec0-fbe5483ece22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:04.196809 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643fc76-757b-4c9e-8ec0-fbe5483ece22 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:04.197096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643fc76-757b-4c9e-8ec0-fbe5483ece22 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:04.197376 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643fc76-757b-4c9e-8ec0-fbe5483ece22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:04.208669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643fc76-757b-4c9e-8ec0-fbe5483ece22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:04.210175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643fc76-757b-4c9e-8ec0-fbe5483ece22 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:04.227698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196f791-bebc-4fbd-8221-e4b580c11711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:04.227874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-0643fc76-757b-4c9e-8ec0-fbe5483ece22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:54:04.227874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:04.227874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:04.227874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:04.227874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:04.227874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:04.227874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:04.227874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:04.227874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:04.227874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:04.227874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:04.227874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:04.227874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:54:04.227874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:54:04.227874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:54:04.228751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:54:04.228751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:04.228751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:04.228751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:04.228751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:04.228751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:04.228751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:04.228751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:04.228751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:04.228751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:04.228751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:04.228751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:04.228751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:04.228751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:04.228751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:04.228751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:04.228751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:04.229653 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:54:04.229653 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:54:04.229653 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:54:04.229653 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:54:04.229895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196f791-bebc-4fbd-8221-e4b580c11711 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:04.229895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196f791-bebc-4fbd-8221-e4b580c11711 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:04.229895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196f791-bebc-4fbd-8221-e4b580c11711 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:04.230090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 554/1096] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:54:04.230158 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196f791-bebc-4fbd-8221-e4b580c11711 None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.project.id=a61a6dd1841c420aae780d467411681f, user.id=544fe6c9f42a43f382f3267e47dcb23a) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:04.230158 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196f791-bebc-4fbd-8221-e4b580c11711 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:04.231587 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196f791-bebc-4fbd-8221-e4b580c11711 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:04.234594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643fc76-757b-4c9e-8ec0-fbe5483ece22 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:04.244767 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 543/1097] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:54:04 2022] GET /identity/v3/role_assignments?user.id=544fe6c9f42a43f382f3267e47dcb23a&scope.project.id=a61a6dd1841c420aae780d467411681f&role.id=053d1730ed764973abb1490894e07681 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:54:04.288812 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:04.363421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643fc76-757b-4c9e-8ec0-fbe5483ece22 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:04.408010 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5118f96-0e6d-4fa5-a2cb-7d586957869e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:04.408682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5118f96-0e6d-4fa5-a2cb-7d586957869e None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:04.408996 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5118f96-0e6d-4fa5-a2cb-7d586957869e None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:04.409282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5118f96-0e6d-4fa5-a2cb-7d586957869e None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:04.410017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5118f96-0e6d-4fa5-a2cb-7d586957869e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:04.410329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5118f96-0e6d-4fa5-a2cb-7d586957869e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:04.411752 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5118f96-0e6d-4fa5-a2cb-7d586957869e None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:04.423974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 555/1098] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:54:04.429546 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196f791-bebc-4fbd-8221-e4b580c11711 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:04.513857 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:04.532851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196f791-bebc-4fbd-8221-e4b580c11711 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:04.565691 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b807d-5468-4e49-8f0c-831509dd7435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:04.566752 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b807d-5468-4e49-8f0c-831509dd7435 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:04.567030 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b807d-5468-4e49-8f0c-831509dd7435 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:04.567298 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b807d-5468-4e49-8f0c-831509dd7435 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:04.575519 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b807d-5468-4e49-8f0c-831509dd7435 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:04.577004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b807d-5468-4e49-8f0c-831509dd7435 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:04.590805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-457b807d-5468-4e49-8f0c-831509dd7435 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:54:04.590805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:04.590805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:04.590805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:04.590805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:04.590805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:04.590805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:04.590805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:04.590805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:04.590805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:04.590805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:04.590805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:04.590805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:54:04.590805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:54:04.590805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:54:04.590805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:04.591832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:04.592837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:04.592837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:04.592837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:54:04.592837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:54:04.592837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:54:04.592837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:54:04.592837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:54:04.592837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:54:04.592837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 21:54:04.592837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:54:04.592837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 544/1099] 10.222.0.44 () {56 vars in 1093 bytes} [Tue Jul 12 21:54:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:54:04.597849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5118f96-0e6d-4fa5-a2cb-7d586957869e None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:04.701888 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5118f96-0e6d-4fa5-a2cb-7d586957869e None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:04.714344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:04.742987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269c033-8d94-49cc-81ee-81ecd775d3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:04.743671 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269c033-8d94-49cc-81ee-81ecd775d3dc None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:04.743954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269c033-8d94-49cc-81ee-81ecd775d3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:04.744232 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269c033-8d94-49cc-81ee-81ecd775d3dc None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:04.745119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269c033-8d94-49cc-81ee-81ecd775d3dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:04.745432 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269c033-8d94-49cc-81ee-81ecd775d3dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:04.746867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269c033-8d94-49cc-81ee-81ecd775d3dc None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:04.764932 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 556/1100] 10.222.0.44 () {56 vars in 1133 bytes} [Tue Jul 12 21:54:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:04.770610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b807d-5468-4e49-8f0c-831509dd7435 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:04.870631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b807d-5468-4e49-8f0c-831509dd7435 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:04.905093 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d36115-5751-4e71-8f8d-cbbe95bad705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:04.905736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d36115-5751-4e71-8f8d-cbbe95bad705 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:04.906017 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d36115-5751-4e71-8f8d-cbbe95bad705 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:04.906292 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d36115-5751-4e71-8f8d-cbbe95bad705 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:04.916584 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:04.977820 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d36115-5751-4e71-8f8d-cbbe95bad705 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:04.979270 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d36115-5751-4e71-8f8d-cbbe95bad705 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:05.083178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 545/1101] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:54:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:54:05.089561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269c033-8d94-49cc-81ee-81ecd775d3dc None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:05.188388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269c033-8d94-49cc-81ee-81ecd775d3dc None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:05.222922 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846055d0-d929-452b-9c1c-d01b2a7fb49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:05.223622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846055d0-d929-452b-9c1c-d01b2a7fb49e None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:05.223905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846055d0-d929-452b-9c1c-d01b2a7fb49e None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:05.224216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846055d0-d929-452b-9c1c-d01b2a7fb49e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:05.234263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846055d0-d929-452b-9c1c-d01b2a7fb49e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:05.235710 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846055d0-d929-452b-9c1c-d01b2a7fb49e None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:05.253113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-846055d0-d929-452b-9c1c-d01b2a7fb49e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:54:05.253113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:05.253113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:05.253113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:05.253113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:05.253113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:05.253113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:05.253113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:05.253113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:05.253113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:05.253113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:05.253113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:05.253113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:54:05.253113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:54:05.253113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:54:05.254220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:54:05.254220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:05.254220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:05.254220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:05.254220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:05.254220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:05.254220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:05.254220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:05.254220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:05.254220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:05.254220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:05.254220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:05.254220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:05.254220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:05.254220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:05.254220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:05.254220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:05.255146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:54:05.255146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:54:05.255146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:54:05.255146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:54:05.255146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 557/1102] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:54:05.260544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d36115-5751-4e71-8f8d-cbbe95bad705 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:05.300404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:05.358016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d36115-5751-4e71-8f8d-cbbe95bad705 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:05.392800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e6393-8653-4bee-b68e-ea66667270e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:05.393464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e6393-8653-4bee-b68e-ea66667270e6 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:05.393748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e6393-8653-4bee-b68e-ea66667270e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:05.394025 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e6393-8653-4bee-b68e-ea66667270e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:05.394743 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e6393-8653-4bee-b68e-ea66667270e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:05.395053 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e6393-8653-4bee-b68e-ea66667270e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:05.396473 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e6393-8653-4bee-b68e-ea66667270e6 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:05.411201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 546/1103] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:05.416916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846055d0-d929-452b-9c1c-d01b2a7fb49e None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:05.436136 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:05.520191 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846055d0-d929-452b-9c1c-d01b2a7fb49e None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:05.553351 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f853dc6-e62e-423a-96a1-e9f431b93f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:05.554130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f853dc6-e62e-423a-96a1-e9f431b93f45 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:05.554490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f853dc6-e62e-423a-96a1-e9f431b93f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:05.554687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f853dc6-e62e-423a-96a1-e9f431b93f45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:05.561180 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f853dc6-e62e-423a-96a1-e9f431b93f45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:05.562622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f853dc6-e62e-423a-96a1-e9f431b93f45 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:05.576847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-4f853dc6-e62e-423a-96a1-e9f431b93f45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:54:05.576847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:05.576847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:05.576847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:05.576847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:05.576847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:05.576847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:05.576847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:05.576847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:05.576847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:05.576847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:05.576847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:05.576847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:54:05.576847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:54:05.576847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:54:05.577976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:54:05.577976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:05.577976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:05.577976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:05.577976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:05.577976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:05.577976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:05.577976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:05.577976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:05.577976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:05.577976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:05.577976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:05.577976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:05.577976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:05.577976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:05.577976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:05.577976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:05.583728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:54:05.583728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:54:05.583728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:54:05.583728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:54:05.583728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:54:05.583728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:54:05.583728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:54:05.583728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:54:05.583728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 558/1104] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:54:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:54:05.584229 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e6393-8653-4bee-b68e-ea66667270e6 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:05.681624 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:05.685329 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e6393-8653-4bee-b68e-ea66667270e6 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:05.718851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83d47f-2c30-426b-8316-1fee755623fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:05.719503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83d47f-2c30-426b-8316-1fee755623fc None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:05.719784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83d47f-2c30-426b-8316-1fee755623fc None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:05.720058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83d47f-2c30-426b-8316-1fee755623fc None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:05.720871 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83d47f-2c30-426b-8316-1fee755623fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:05.721179 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83d47f-2c30-426b-8316-1fee755623fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:05.722593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83d47f-2c30-426b-8316-1fee755623fc None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:05.734876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 547/1105] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:54:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:05.741474 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f853dc6-e62e-423a-96a1-e9f431b93f45 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:05.819393 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:05.840873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f853dc6-e62e-423a-96a1-e9f431b93f45 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:05.874068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee980f-c2a9-46c6-a74e-93d8f1832b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:05.874710 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee980f-c2a9-46c6-a74e-93d8f1832b01 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:05.874991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee980f-c2a9-46c6-a74e-93d8f1832b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:05.875262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee980f-c2a9-46c6-a74e-93d8f1832b01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:05.876085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee980f-c2a9-46c6-a74e-93d8f1832b01 None admin] RBAC: Adding query filter params (role.id=3ee4a924b5464226903b6ad15ad59ebc, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=24915625354f418eb7bc4d5ada85fdb0) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:05.876392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee980f-c2a9-46c6-a74e-93d8f1832b01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:05.877863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee980f-c2a9-46c6-a74e-93d8f1832b01 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:05.884190 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 559/1106] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:54:05 2022] GET /identity/v3/role_assignments?user.id=24915625354f418eb7bc4d5ada85fdb0&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=3ee4a924b5464226903b6ad15ad59ebc => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:06.388378 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148948c-0f88-42bb-be0d-c5a121b661c9 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:06.388950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148948c-0f88-42bb-be0d-c5a121b661c9 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:06.389046 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148948c-0f88-42bb-be0d-c5a121b661c9 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:06.390353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 548/1107] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:06.395630 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da210044-1cef-4f5e-afe0-3adb86ab622b None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:06.396097 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da210044-1cef-4f5e-afe0-3adb86ab622b None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:06.396380 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da210044-1cef-4f5e-afe0-3adb86ab622b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:06.414250 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-da210044-1cef-4f5e-afe0-3adb86ab622b None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:06.415447 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-da210044-1cef-4f5e-afe0-3adb86ab622b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:06.485762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 560/1108] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:06.510843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148948c-0f88-42bb-be0d-c5a121b661c9 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:06.616634 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148948c-0f88-42bb-be0d-c5a121b661c9 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:06.652876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddaf4b-b7cc-456f-bc2d-3e0c250269e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:06.653607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddaf4b-b7cc-456f-bc2d-3e0c250269e3 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:06.653883 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddaf4b-b7cc-456f-bc2d-3e0c250269e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:06.654162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddaf4b-b7cc-456f-bc2d-3e0c250269e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:06.655341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddaf4b-b7cc-456f-bc2d-3e0c250269e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:06.656818 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddaf4b-b7cc-456f-bc2d-3e0c250269e3 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:06.658603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 549/1109] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:54:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:06.664312 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da210044-1cef-4f5e-afe0-3adb86ab622b None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:06.737082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:06.769757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da210044-1cef-4f5e-afe0-3adb86ab622b None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:06.803954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefaf5f-6bb3-4903-8777-6552e682b4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:06.804614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefaf5f-6bb3-4903-8777-6552e682b4ef None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:06.804926 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefaf5f-6bb3-4903-8777-6552e682b4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:06.805204 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefaf5f-6bb3-4903-8777-6552e682b4ef None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:06.805938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefaf5f-6bb3-4903-8777-6552e682b4ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:06.807329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefaf5f-6bb3-4903-8777-6552e682b4ef None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:06.827628 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 561/1110] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:54:06 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:54:08.921592 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6d4cc-ab03-4613-9fd1-5258ea48ca67 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:08.922156 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6d4cc-ab03-4613-9fd1-5258ea48ca67 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:08.922156 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6d4cc-ab03-4613-9fd1-5258ea48ca67 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:08.923542 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 550/1111] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:08.928766 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac941324-1b4a-4f3f-9f7b-7b1403c86296 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:08.929178 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac941324-1b4a-4f3f-9f7b-7b1403c86296 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:08.929336 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac941324-1b4a-4f3f-9f7b-7b1403c86296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:08.951288 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-ac941324-1b4a-4f3f-9f7b-7b1403c86296 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:08.952500 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ac941324-1b4a-4f3f-9f7b-7b1403c86296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:09.019849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 562/1112] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:09.045043 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6d4cc-ab03-4613-9fd1-5258ea48ca67 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:09.073011 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:09.162720 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6d4cc-ab03-4613-9fd1-5258ea48ca67 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:09.195681 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741a8f9-db03-4a28-b229-e69477bfb01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:09.196458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741a8f9-db03-4a28-b229-e69477bfb01e None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:09.196766 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741a8f9-db03-4a28-b229-e69477bfb01e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:09.197042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741a8f9-db03-4a28-b229-e69477bfb01e None admin] PATH_INFO: `/v3/roles/160d4162340e472b9712a7f15cb2ecca` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:09.198481 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741a8f9-db03-4a28-b229-e69477bfb01e None admin] RBAC: Authorizing `identity:get_role(role_id=160d4162340e472b9712a7f15cb2ecca)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:09.199856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741a8f9-db03-4a28-b229-e69477bfb01e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:09.201250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 551/1113] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:54:09 2022] GET /identity/v3/roles/160d4162340e472b9712a7f15cb2ecca => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:09.207185 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac941324-1b4a-4f3f-9f7b-7b1403c86296 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:09.266126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:09.311661 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac941324-1b4a-4f3f-9f7b-7b1403c86296 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:09.346082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3fe0f-83c3-4c66-9b7f-dc61f6838df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:09.346946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3fe0f-83c3-4c66-9b7f-dc61f6838df9 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:09.347230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3fe0f-83c3-4c66-9b7f-dc61f6838df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:09.347507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3fe0f-83c3-4c66-9b7f-dc61f6838df9 None admin] PATH_INFO: `/v3/users/e10f836c12354d9a8533ff2dfb22d5d8` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:09.362528 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3fe0f-83c3-4c66-9b7f-dc61f6838df9 None admin] RBAC: Authorizing `identity:get_user(user_id=e10f836c12354d9a8533ff2dfb22d5d8)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:09.363976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3fe0f-83c3-4c66-9b7f-dc61f6838df9 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:09.366116 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 563/1114] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:54:09 2022] GET /identity/v3/users/e10f836c12354d9a8533ff2dfb22d5d8 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:09.371934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741a8f9-db03-4a28-b229-e69477bfb01e None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:09.476139 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:09.482383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741a8f9-db03-4a28-b229-e69477bfb01e None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:09.515864 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2d132-79f9-4ec5-8613-38c054818f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:09.516512 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2d132-79f9-4ec5-8613-38c054818f70 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:09.516824 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2d132-79f9-4ec5-8613-38c054818f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:09.517104 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2d132-79f9-4ec5-8613-38c054818f70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:09.583172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c2d132-79f9-4ec5-8613-38c054818f70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:09.584633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c2d132-79f9-4ec5-8613-38c054818f70 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:09.679515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:09.696623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 552/1115] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:54:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:54:09.702670 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3fe0f-83c3-4c66-9b7f-dc61f6838df9 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:09.802849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3fe0f-83c3-4c66-9b7f-dc61f6838df9 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:09.819046 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:09.837796 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05ca32-8724-4fc9-9022-95a082f7ad72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:09.838658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05ca32-8724-4fc9-9022-95a082f7ad72 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:09.838942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05ca32-8724-4fc9-9022-95a082f7ad72 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:09.839220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05ca32-8724-4fc9-9022-95a082f7ad72 None admin] PATH_INFO: `/v3/projects/a61a6dd1841c420aae780d467411681f` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:09.840386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05ca32-8724-4fc9-9022-95a082f7ad72 None admin] RBAC: Authorizing `identity:get_project(project_id=a61a6dd1841c420aae780d467411681f)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:09.841872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05ca32-8724-4fc9-9022-95a082f7ad72 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:09.843579 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 564/1116] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:54:09 2022] GET /identity/v3/projects/a61a6dd1841c420aae780d467411681f => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:09.850193 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2d132-79f9-4ec5-8613-38c054818f70 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:09.949892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2d132-79f9-4ec5-8613-38c054818f70 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:09.984685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735cea94-cf83-4d72-9497-bc94f41aac19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:09.985363 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cea94-cf83-4d72-9497-bc94f41aac19 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:09.985651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cea94-cf83-4d72-9497-bc94f41aac19 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:09.985931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cea94-cf83-4d72-9497-bc94f41aac19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:09.986757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735cea94-cf83-4d72-9497-bc94f41aac19 None admin] RBAC: Adding query filter params (role.id=160d4162340e472b9712a7f15cb2ecca, scope.project.id=a61a6dd1841c420aae780d467411681f, user.id=e10f836c12354d9a8533ff2dfb22d5d8) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:09.987074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735cea94-cf83-4d72-9497-bc94f41aac19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:09.988516 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735cea94-cf83-4d72-9497-bc94f41aac19 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:09.993948 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 553/1117] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:54:09 2022] GET /identity/v3/role_assignments?user.id=e10f836c12354d9a8533ff2dfb22d5d8&scope.project.id=a61a6dd1841c420aae780d467411681f&role.id=160d4162340e472b9712a7f15cb2ecca => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:12.002550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ec41b-b776-43f9-ad7e-94f028245d01 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:12.003057 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ec41b-b776-43f9-ad7e-94f028245d01 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:12.003158 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ec41b-b776-43f9-ad7e-94f028245d01 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:12.004458 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 565/1118] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:12.010402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffac7a-9388-4841-b621-c6c439c7c74a None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:12.010879 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffac7a-9388-4841-b621-c6c439c7c74a None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:12.010982 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffac7a-9388-4841-b621-c6c439c7c74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:12.028469 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-a9ffac7a-9388-4841-b621-c6c439c7c74a None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:12.029718 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a9ffac7a-9388-4841-b621-c6c439c7c74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:12.085566 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:12.097053 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 554/1119] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:12.120164 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ec41b-b776-43f9-ad7e-94f028245d01 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:12.219064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ec41b-b776-43f9-ad7e-94f028245d01 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:12.252637 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e5840-6e12-4a9d-9999-cd12f0226eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:12.253517 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e5840-6e12-4a9d-9999-cd12f0226eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:12.253803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e5840-6e12-4a9d-9999-cd12f0226eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:12.254080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e5840-6e12-4a9d-9999-cd12f0226eb4 None admin] PATH_INFO: `/v3/roles/160d4162340e472b9712a7f15cb2ecca` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:12.256158 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e5840-6e12-4a9d-9999-cd12f0226eb4 None admin] RBAC: Authorizing `identity:get_role(role_id=160d4162340e472b9712a7f15cb2ecca)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:12.257570 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e5840-6e12-4a9d-9999-cd12f0226eb4 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:12.258945 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 566/1120] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:54:12 2022] GET /identity/v3/roles/160d4162340e472b9712a7f15cb2ecca => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:12.264768 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffac7a-9388-4841-b621-c6c439c7c74a None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:12.362053 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffac7a-9388-4841-b621-c6c439c7c74a None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:12.395872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd621f3-5b35-45c0-8163-dd744efefb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:12.396603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd621f3-5b35-45c0-8163-dd744efefb26 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:12.396915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd621f3-5b35-45c0-8163-dd744efefb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:12.397192 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd621f3-5b35-45c0-8163-dd744efefb26 None admin] PATH_INFO: `/v3/users/e10f836c12354d9a8533ff2dfb22d5d8` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:12.398363 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd621f3-5b35-45c0-8163-dd744efefb26 None admin] RBAC: Authorizing `identity:get_user(user_id=e10f836c12354d9a8533ff2dfb22d5d8)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:12.399797 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd621f3-5b35-45c0-8163-dd744efefb26 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:12.401462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 555/1121] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:54:12 2022] GET /identity/v3/users/e10f836c12354d9a8533ff2dfb22d5d8 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:12.407296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e5840-6e12-4a9d-9999-cd12f0226eb4 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:12.459324 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:12.506534 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e5840-6e12-4a9d-9999-cd12f0226eb4 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:12.540163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214f338-da4f-4faf-9a28-4b1c46db46c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:12.541050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214f338-da4f-4faf-9a28-4b1c46db46c8 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:12.541337 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214f338-da4f-4faf-9a28-4b1c46db46c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:12.541618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214f338-da4f-4faf-9a28-4b1c46db46c8 None admin] PATH_INFO: `/v3/projects/a61a6dd1841c420aae780d467411681f` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:12.542797 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214f338-da4f-4faf-9a28-4b1c46db46c8 None admin] RBAC: Authorizing `identity:get_project(project_id=a61a6dd1841c420aae780d467411681f)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:12.544243 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214f338-da4f-4faf-9a28-4b1c46db46c8 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:12.545986 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 567/1122] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:54:12 2022] GET /identity/v3/projects/a61a6dd1841c420aae780d467411681f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:12.552578 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd621f3-5b35-45c0-8163-dd744efefb26 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:12.608969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:12.650430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd621f3-5b35-45c0-8163-dd744efefb26 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:12.683888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe72d3-2aa9-4a6b-9aa9-1c6bfd1c8a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:12.684787 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe72d3-2aa9-4a6b-9aa9-1c6bfd1c8a34 None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:12.685073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe72d3-2aa9-4a6b-9aa9-1c6bfd1c8a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:12.685353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe72d3-2aa9-4a6b-9aa9-1c6bfd1c8a34 None admin] PATH_INFO: `/v3/projects/a61a6dd1841c420aae780d467411681f/users/e10f836c12354d9a8533ff2dfb22d5d8/roles/160d4162340e472b9712a7f15cb2ecca` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:12.687525 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe72d3-2aa9-4a6b-9aa9-1c6bfd1c8a34 None admin] RBAC: Authorizing `identity:create_grant(project_id=a61a6dd1841c420aae780d467411681f, user_id=e10f836c12354d9a8533ff2dfb22d5d8, role_id=160d4162340e472b9712a7f15cb2ecca)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:12.688984 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe72d3-2aa9-4a6b-9aa9-1c6bfd1c8a34 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:12.702119 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 556/1123] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:54:12 2022] PUT /identity/v3/projects/a61a6dd1841c420aae780d467411681f/users/e10f836c12354d9a8533ff2dfb22d5d8/roles/160d4162340e472b9712a7f15cb2ecca => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:54:14.719008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9a747-5a2b-4900-b035-910a14621000 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:14.719546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9a747-5a2b-4900-b035-910a14621000 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:14.719546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9a747-5a2b-4900-b035-910a14621000 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:14.720979 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 568/1124] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:14.726287 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87750d-48cc-46bc-9740-b18fd8fb5e33 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:14.726697 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87750d-48cc-46bc-9740-b18fd8fb5e33 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:14.726851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87750d-48cc-46bc-9740-b18fd8fb5e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:14.743280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-6a87750d-48cc-46bc-9740-b18fd8fb5e33 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:14.744467 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6a87750d-48cc-46bc-9740-b18fd8fb5e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:14.809013 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 557/1125] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:14.833883 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9a747-5a2b-4900-b035-910a14621000 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:14.934438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9a747-5a2b-4900-b035-910a14621000 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:14.954680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:14.969816 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c411a9-275d-495f-8703-d666d46526ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:14.970616 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c411a9-275d-495f-8703-d666d46526ef None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:14.970901 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c411a9-275d-495f-8703-d666d46526ef None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:14.971180 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c411a9-275d-495f-8703-d666d46526ef None admin] PATH_INFO: `/v3/roles/160d4162340e472b9712a7f15cb2ecca` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:14.972659 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c411a9-275d-495f-8703-d666d46526ef None admin] RBAC: Authorizing `identity:get_role(role_id=160d4162340e472b9712a7f15cb2ecca)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:14.974117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c411a9-275d-495f-8703-d666d46526ef None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:14.975623 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 569/1126] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:54:14 2022] GET /identity/v3/roles/160d4162340e472b9712a7f15cb2ecca => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:14.981341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87750d-48cc-46bc-9740-b18fd8fb5e33 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:15.029321 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:15.081190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87750d-48cc-46bc-9740-b18fd8fb5e33 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:15.113708 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1bad7-8eac-45db-8ab6-99f19cd3da1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:15.115060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1bad7-8eac-45db-8ab6-99f19cd3da1c None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:15.115397 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1bad7-8eac-45db-8ab6-99f19cd3da1c None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:15.115612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1bad7-8eac-45db-8ab6-99f19cd3da1c None admin] PATH_INFO: `/v3/users/e10f836c12354d9a8533ff2dfb22d5d8` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:15.116816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1bad7-8eac-45db-8ab6-99f19cd3da1c None admin] RBAC: Authorizing `identity:get_user(user_id=e10f836c12354d9a8533ff2dfb22d5d8)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:15.118260 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1bad7-8eac-45db-8ab6-99f19cd3da1c None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:15.119889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 558/1127] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:54:14 2022] GET /identity/v3/users/e10f836c12354d9a8533ff2dfb22d5d8 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:15.125692 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c411a9-275d-495f-8703-d666d46526ef None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:15.223799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c411a9-275d-495f-8703-d666d46526ef None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:15.257876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e797ec0-d890-4577-99a4-e64c1b6ef2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:15.258521 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e797ec0-d890-4577-99a4-e64c1b6ef2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:15.258807 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e797ec0-d890-4577-99a4-e64c1b6ef2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:15.259083 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e797ec0-d890-4577-99a4-e64c1b6ef2d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:15.312018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:15.325415 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e797ec0-d890-4577-99a4-e64c1b6ef2d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:15.326865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e797ec0-d890-4577-99a4-e64c1b6ef2d7 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:15.422384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 570/1128] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:54:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:54:15.429010 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1bad7-8eac-45db-8ab6-99f19cd3da1c None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:15.528132 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1bad7-8eac-45db-8ab6-99f19cd3da1c None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:15.544883 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:15.563198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a1f0c-d5ff-4d6e-8551-391099edae18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:15.564052 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a1f0c-d5ff-4d6e-8551-391099edae18 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:15.564334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a1f0c-d5ff-4d6e-8551-391099edae18 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:15.564609 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a1f0c-d5ff-4d6e-8551-391099edae18 None admin] PATH_INFO: `/v3/projects/a61a6dd1841c420aae780d467411681f` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:15.565826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a1f0c-d5ff-4d6e-8551-391099edae18 None admin] RBAC: Authorizing `identity:get_project(project_id=a61a6dd1841c420aae780d467411681f)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:15.567870 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a1f0c-d5ff-4d6e-8551-391099edae18 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:15.569762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 559/1129] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:54:15 2022] GET /identity/v3/projects/a61a6dd1841c420aae780d467411681f => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:15.575827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e797ec0-d890-4577-99a4-e64c1b6ef2d7 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:15.660540 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:15.672864 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e797ec0-d890-4577-99a4-e64c1b6ef2d7 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:15.706599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80935029-d16b-4259-8b80-462122a39fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:15.707237 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80935029-d16b-4259-8b80-462122a39fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:15.707518 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80935029-d16b-4259-8b80-462122a39fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:15.707791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80935029-d16b-4259-8b80-462122a39fe1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:15.709360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80935029-d16b-4259-8b80-462122a39fe1 None admin] RBAC: Adding query filter params (role.id=160d4162340e472b9712a7f15cb2ecca, scope.project.id=a61a6dd1841c420aae780d467411681f, user.id=e10f836c12354d9a8533ff2dfb22d5d8) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:15.709654 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80935029-d16b-4259-8b80-462122a39fe1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:15.711065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80935029-d16b-4259-8b80-462122a39fe1 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:15.717383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 571/1130] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:54:15 2022] GET /identity/v3/role_assignments?user.id=e10f836c12354d9a8533ff2dfb22d5d8&scope.project.id=a61a6dd1841c420aae780d467411681f&role.id=160d4162340e472b9712a7f15cb2ecca => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:17.742143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1574b-33cd-43e7-a291-4e480dee0f7f None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:17.742674 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1574b-33cd-43e7-a291-4e480dee0f7f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:17.742800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1574b-33cd-43e7-a291-4e480dee0f7f None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:17.744070 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 560/1131] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:17.749356 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b23d07-df28-4a03-ac4d-da79dd35e621 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:17.749796 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b23d07-df28-4a03-ac4d-da79dd35e621 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:17.749898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b23d07-df28-4a03-ac4d-da79dd35e621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:17.766926 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-08b23d07-df28-4a03-ac4d-da79dd35e621 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:17.768091 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-08b23d07-df28-4a03-ac4d-da79dd35e621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:17.834669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 572/1132] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:17.857653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1574b-33cd-43e7-a291-4e480dee0f7f None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:17.957249 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1574b-33cd-43e7-a291-4e480dee0f7f None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:17.993181 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b9298-a831-4114-b044-ed3ab39301b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:17.993908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b9298-a831-4114-b044-ed3ab39301b0 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:17.994193 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b9298-a831-4114-b044-ed3ab39301b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:17.994473 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b9298-a831-4114-b044-ed3ab39301b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:17.995639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b9298-a831-4114-b044-ed3ab39301b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:17.997103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b9298-a831-4114-b044-ed3ab39301b0 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:17.998748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 561/1133] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:54:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:18.004656 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b23d07-df28-4a03-ac4d-da79dd35e621 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:18.082637 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:18.101687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b23d07-df28-4a03-ac4d-da79dd35e621 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:18.137804 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ee965-9b67-4c0a-863a-5e80f4927909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:18.138457 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ee965-9b67-4c0a-863a-5e80f4927909 None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:18.138736 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ee965-9b67-4c0a-863a-5e80f4927909 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:18.139008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ee965-9b67-4c0a-863a-5e80f4927909 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:18.139711 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ee965-9b67-4c0a-863a-5e80f4927909 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:18.141164 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ee965-9b67-4c0a-863a-5e80f4927909 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:18.165573 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 573/1134] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Jul 12 21:54:18 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:54:20.249100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9f171-4cce-4056-8f23-1bffccf80765 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:20.249625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9f171-4cce-4056-8f23-1bffccf80765 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:20.249625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9f171-4cce-4056-8f23-1bffccf80765 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:20.250925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 562/1135] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:20.257026 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c28d8c-48e8-4a9c-b583-0c2cc36a5771 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:20.257439 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c28d8c-48e8-4a9c-b583-0c2cc36a5771 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:20.257577 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c28d8c-48e8-4a9c-b583-0c2cc36a5771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:20.274021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-02c28d8c-48e8-4a9c-b583-0c2cc36a5771 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:20.275212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-02c28d8c-48e8-4a9c-b583-0c2cc36a5771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:20.344697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 574/1136] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:20.369773 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9f171-4cce-4056-8f23-1bffccf80765 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:20.422945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:20.469152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9f171-4cce-4056-8f23-1bffccf80765 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:20.505469 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b4f22-9618-4e79-a921-a5ffe8156f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:20.506244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b4f22-9618-4e79-a921-a5ffe8156f26 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:20.506527 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b4f22-9618-4e79-a921-a5ffe8156f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:20.506800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b4f22-9618-4e79-a921-a5ffe8156f26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:20.507981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b4f22-9618-4e79-a921-a5ffe8156f26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:20.509446 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b4f22-9618-4e79-a921-a5ffe8156f26 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:20.511057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 563/1137] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:54:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:54:20.517334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c28d8c-48e8-4a9c-b583-0c2cc36a5771 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:20.580436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:20.619004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c28d8c-48e8-4a9c-b583-0c2cc36a5771 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:20.652989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4cbe7-d151-45b8-9418-6a4e495e11eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:20.653655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4cbe7-d151-45b8-9418-6a4e495e11eb None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:20.653938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4cbe7-d151-45b8-9418-6a4e495e11eb None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:20.654214 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4cbe7-d151-45b8-9418-6a4e495e11eb None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:20.654932 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4cbe7-d151-45b8-9418-6a4e495e11eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:20.656394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4cbe7-d151-45b8-9418-6a4e495e11eb None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:20.675076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 575/1138] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:54:20 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:54:22.723288 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921f193-5fef-42f2-b75c-2ed327524c0b None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:22.724021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921f193-5fef-42f2-b75c-2ed327524c0b None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:22.724021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921f193-5fef-42f2-b75c-2ed327524c0b None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:22.725318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 564/1139] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:22.730548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36969404-5995-403c-8f51-4f7ee0a94070 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:22.730981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36969404-5995-403c-8f51-4f7ee0a94070 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:22.731113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36969404-5995-403c-8f51-4f7ee0a94070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:22.748024 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-36969404-5995-403c-8f51-4f7ee0a94070 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:22.749238 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-36969404-5995-403c-8f51-4f7ee0a94070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:22.814826 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 576/1140] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:22.839443 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921f193-5fef-42f2-b75c-2ed327524c0b None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:22.947244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921f193-5fef-42f2-b75c-2ed327524c0b None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:22.967200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:22.982268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86787800-7297-4333-a8a2-282d6309f2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:22.982998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86787800-7297-4333-a8a2-282d6309f2ac None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:22.983281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86787800-7297-4333-a8a2-282d6309f2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:22.983555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86787800-7297-4333-a8a2-282d6309f2ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:22.994180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86787800-7297-4333-a8a2-282d6309f2ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:54:23.001755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-86787800-7297-4333-a8a2-282d6309f2ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:54:23.001755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:23.001755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:23.001755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:23.001755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:23.001755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:23.001755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:23.001755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:23.001755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:23.001755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:23.001755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:23.001755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:23.001755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:54:23.001755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:54:23.001755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:54:23.001755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:54:23.002924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:54:23.002924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:54:23.002924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:54:23.002924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:54:23.002924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:23.002924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:23.002924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:23.002924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:23.002924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:23.002924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:23.002924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:23.002924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:23.002924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:23.002924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:23.002924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:23.002924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:23.004178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:23.004178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:23.004178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:23.004178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:23.004178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:54:23.004178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:54:23.004178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:54:23.004178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:54:23.004178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:54:23.004178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:54:23.004178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:54:23.004178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:54:23.004178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 565/1141] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:54:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:54:23.009232 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36969404-5995-403c-8f51-4f7ee0a94070 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:23.057676 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:23.109550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36969404-5995-403c-8f51-4f7ee0a94070 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:23.143420 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb14bec-9af3-47dd-a0cf-758cc88e6b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:23.144091 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb14bec-9af3-47dd-a0cf-758cc88e6b83 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:23.144380 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb14bec-9af3-47dd-a0cf-758cc88e6b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:23.144657 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb14bec-9af3-47dd-a0cf-758cc88e6b83 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:23.146112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb14bec-9af3-47dd-a0cf-758cc88e6b83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:23.146424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb14bec-9af3-47dd-a0cf-758cc88e6b83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:23.147808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb14bec-9af3-47dd-a0cf-758cc88e6b83 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:23.156144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 577/1142] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:54:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:23.162103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86787800-7297-4333-a8a2-282d6309f2ac None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:23.261723 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86787800-7297-4333-a8a2-282d6309f2ac None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:23.293914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876596d4-e623-47ab-915d-64e52b14ef99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:23.294772 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876596d4-e623-47ab-915d-64e52b14ef99 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:23.295054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876596d4-e623-47ab-915d-64e52b14ef99 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:23.295331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876596d4-e623-47ab-915d-64e52b14ef99 None admin] PATH_INFO: `/v3/users/7fe8c3e113134177ac1f6ba224e21b25` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:23.308651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876596d4-e623-47ab-915d-64e52b14ef99 None admin] RBAC: Authorizing `identity:get_user(user_id=7fe8c3e113134177ac1f6ba224e21b25)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:23.310129 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876596d4-e623-47ab-915d-64e52b14ef99 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:23.311783 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 566/1143] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:54:23 2022] GET /identity/v3/users/7fe8c3e113134177ac1f6ba224e21b25 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:23.318479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb14bec-9af3-47dd-a0cf-758cc88e6b83 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:23.414305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb14bec-9af3-47dd-a0cf-758cc88e6b83 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:23.426958 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:23.448900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d77ea-463b-4adb-94d1-542b47ec5867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:23.449544 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d77ea-463b-4adb-94d1-542b47ec5867 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:23.449829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d77ea-463b-4adb-94d1-542b47ec5867 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:23.450111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d77ea-463b-4adb-94d1-542b47ec5867 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:23.515842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d77ea-463b-4adb-94d1-542b47ec5867 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:23.517273 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d77ea-463b-4adb-94d1-542b47ec5867 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:23.613594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 578/1144] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:54:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:54:23.620196 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876596d4-e623-47ab-915d-64e52b14ef99 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:23.659807 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:23.717360 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876596d4-e623-47ab-915d-64e52b14ef99 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:23.752403 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f70d7-7ae9-4f20-b1e6-f03bb6132879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:23.753285 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f70d7-7ae9-4f20-b1e6-f03bb6132879 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:23.753568 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f70d7-7ae9-4f20-b1e6-f03bb6132879 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:23.753841 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f70d7-7ae9-4f20-b1e6-f03bb6132879 None admin] PATH_INFO: `/v3/projects/195e7970834244b9bfa0fdffda0548fd` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:23.755021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f70d7-7ae9-4f20-b1e6-f03bb6132879 None admin] RBAC: Authorizing `identity:get_project(project_id=195e7970834244b9bfa0fdffda0548fd)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:23.756447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f70d7-7ae9-4f20-b1e6-f03bb6132879 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:23.758172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 567/1145] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:54:23 2022] GET /identity/v3/projects/195e7970834244b9bfa0fdffda0548fd => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:23.764239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d77ea-463b-4adb-94d1-542b47ec5867 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:23.778469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:23.861391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d77ea-463b-4adb-94d1-542b47ec5867 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:23.894344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc3c5a-d34b-4aee-9b0a-0425ab923e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:23.894980 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc3c5a-d34b-4aee-9b0a-0425ab923e0a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:23.895304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc3c5a-d34b-4aee-9b0a-0425ab923e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:23.895621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc3c5a-d34b-4aee-9b0a-0425ab923e0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:23.896444 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc3c5a-d34b-4aee-9b0a-0425ab923e0a None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.project.id=195e7970834244b9bfa0fdffda0548fd, user.id=7fe8c3e113134177ac1f6ba224e21b25) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:23.896773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc3c5a-d34b-4aee-9b0a-0425ab923e0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:23.898198 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc3c5a-d34b-4aee-9b0a-0425ab923e0a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:23.903515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 579/1146] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:54:23 2022] GET /identity/v3/role_assignments?user.id=7fe8c3e113134177ac1f6ba224e21b25&scope.project.id=195e7970834244b9bfa0fdffda0548fd&role.id=053d1730ed764973abb1490894e07681 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:25.880253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c790c-da02-4c3b-b3c5-3540ce54284c None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:25.880860 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c790c-da02-4c3b-b3c5-3540ce54284c None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:25.880860 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c790c-da02-4c3b-b3c5-3540ce54284c None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:25.882199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 568/1147] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:25.888563 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082741d-9696-4083-aba0-051319fa152d None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:25.889118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082741d-9696-4083-aba0-051319fa152d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:25.889219 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082741d-9696-4083-aba0-051319fa152d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:25.905544 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-a082741d-9696-4083-aba0-051319fa152d None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:25.906737 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a082741d-9696-4083-aba0-051319fa152d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:25.972948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 580/1148] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:25.996219 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c790c-da02-4c3b-b3c5-3540ce54284c None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:26.095467 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c790c-da02-4c3b-b3c5-3540ce54284c None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:26.103250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:26.130554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905cb878-75ae-41ee-bacd-620fd16e4d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:26.131273 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cb878-75ae-41ee-bacd-620fd16e4d0d None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:26.131558 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cb878-75ae-41ee-bacd-620fd16e4d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:26.131837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cb878-75ae-41ee-bacd-620fd16e4d0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:26.141758 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905cb878-75ae-41ee-bacd-620fd16e4d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:54:26.149286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-905cb878-75ae-41ee-bacd-620fd16e4d0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:54:26.149286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:26.149286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:26.149286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:26.149286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:26.149286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:26.149286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:26.149286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:26.149286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:26.149286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:26.149286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:26.149286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:26.149286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:54:26.149286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:54:26.149286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:54:26.149286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:54:26.150385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:54:26.150385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:54:26.150385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:54:26.150385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:54:26.150385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:26.150385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:26.150385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:26.150385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:26.150385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:26.150385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:26.150385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:26.150385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:26.150385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:26.150385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:26.150385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:26.150385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:26.151596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:26.151596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:26.151596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:26.151596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:26.151596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:54:26.151596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:54:26.151596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:54:26.151596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:54:26.151596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:54:26.151596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:54:26.151596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:54:26.151596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:54:26.151596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 569/1149] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:54:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:54:26.156543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082741d-9696-4083-aba0-051319fa152d None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:26.165277 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:26.255697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082741d-9696-4083-aba0-051319fa152d None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:26.289565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d6125-0319-4de5-b842-b797b92da380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:26.290226 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d6125-0319-4de5-b842-b797b92da380 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:26.290513 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d6125-0319-4de5-b842-b797b92da380 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:26.290796 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d6125-0319-4de5-b842-b797b92da380 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:26.291545 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d6125-0319-4de5-b842-b797b92da380 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:26.291855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d6125-0319-4de5-b842-b797b92da380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:26.293277 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d6125-0319-4de5-b842-b797b92da380 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:26.301226 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 581/1150] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:54:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:26.307121 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905cb878-75ae-41ee-bacd-620fd16e4d0d None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:26.405642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905cb878-75ae-41ee-bacd-620fd16e4d0d None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:26.439365 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e3459-e330-41ac-b192-fe5ac2a2feec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:26.440227 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e3459-e330-41ac-b192-fe5ac2a2feec None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:26.440505 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e3459-e330-41ac-b192-fe5ac2a2feec None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:26.440820 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e3459-e330-41ac-b192-fe5ac2a2feec None admin] PATH_INFO: `/v3/users/7fe8c3e113134177ac1f6ba224e21b25` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:26.442926 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e3459-e330-41ac-b192-fe5ac2a2feec None admin] RBAC: Authorizing `identity:get_user(user_id=7fe8c3e113134177ac1f6ba224e21b25)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:26.444374 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e3459-e330-41ac-b192-fe5ac2a2feec None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:26.446078 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 570/1151] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:54:26 2022] GET /identity/v3/users/7fe8c3e113134177ac1f6ba224e21b25 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:26.451895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d6125-0319-4de5-b842-b797b92da380 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:26.516784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:26.549875 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d6125-0319-4de5-b842-b797b92da380 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:26.583658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199b154-a88c-4a57-85a5-eee7841573b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:26.584539 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199b154-a88c-4a57-85a5-eee7841573b4 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:26.584847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199b154-a88c-4a57-85a5-eee7841573b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:26.585123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199b154-a88c-4a57-85a5-eee7841573b4 None admin] PATH_INFO: `/v3/projects/195e7970834244b9bfa0fdffda0548fd` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:26.586287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199b154-a88c-4a57-85a5-eee7841573b4 None admin] RBAC: Authorizing `identity:get_project(project_id=195e7970834244b9bfa0fdffda0548fd)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:26.587731 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199b154-a88c-4a57-85a5-eee7841573b4 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:26.589454 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 582/1152] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:54:26 2022] GET /identity/v3/projects/195e7970834244b9bfa0fdffda0548fd => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:26.595388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e3459-e330-41ac-b192-fe5ac2a2feec None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:26.629081 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:26.693898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e3459-e330-41ac-b192-fe5ac2a2feec None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:26.725837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca74d8-4fa4-4e8d-aff3-a1387f26ba95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:26.726717 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca74d8-4fa4-4e8d-aff3-a1387f26ba95 None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:26.726992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca74d8-4fa4-4e8d-aff3-a1387f26ba95 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:26.727262 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca74d8-4fa4-4e8d-aff3-a1387f26ba95 None admin] PATH_INFO: `/v3/projects/195e7970834244b9bfa0fdffda0548fd/users/7fe8c3e113134177ac1f6ba224e21b25/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:26.729229 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca74d8-4fa4-4e8d-aff3-a1387f26ba95 None admin] RBAC: Authorizing `identity:create_grant(project_id=195e7970834244b9bfa0fdffda0548fd, user_id=7fe8c3e113134177ac1f6ba224e21b25, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:26.730650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca74d8-4fa4-4e8d-aff3-a1387f26ba95 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:26.743671 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 571/1153] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:54:26 2022] PUT /identity/v3/projects/195e7970834244b9bfa0fdffda0548fd/users/7fe8c3e113134177ac1f6ba224e21b25/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:54:28.771093 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896e3f9-403d-4a78-a69c-900ce4be46c3 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:28.771706 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896e3f9-403d-4a78-a69c-900ce4be46c3 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:28.771947 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896e3f9-403d-4a78-a69c-900ce4be46c3 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:28.773762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 583/1154] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:28.779068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61dd89-112e-4285-b46d-5726e8f2a1fc None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:28.779702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61dd89-112e-4285-b46d-5726e8f2a1fc None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:28.779702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61dd89-112e-4285-b46d-5726e8f2a1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:28.796559 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-de61dd89-112e-4285-b46d-5726e8f2a1fc None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:28.797789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-de61dd89-112e-4285-b46d-5726e8f2a1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:28.864310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 572/1155] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:28.889104 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896e3f9-403d-4a78-a69c-900ce4be46c3 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:28.995961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896e3f9-403d-4a78-a69c-900ce4be46c3 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:29.029328 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24178b-868a-495f-a1fb-1866d89b7bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:29.030047 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24178b-868a-495f-a1fb-1866d89b7bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:29.030332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24178b-868a-495f-a1fb-1866d89b7bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:29.030611 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24178b-868a-495f-a1fb-1866d89b7bd8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:29.033214 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:29.043226 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24178b-868a-495f-a1fb-1866d89b7bd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:54:29.050831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-1c24178b-868a-495f-a1fb-1866d89b7bd8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:54:29.050831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:29.050831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:29.050831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:29.050831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:29.050831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:29.050831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:29.050831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:29.050831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:29.050831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:29.050831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:29.050831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:29.050831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:54:29.050831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:54:29.050831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:54:29.050831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:54:29.051923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:54:29.051923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:54:29.051923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:54:29.051923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:54:29.051923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:29.051923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:29.051923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:29.051923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:29.051923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:29.051923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:29.051923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:29.051923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:29.051923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:29.051923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:29.051923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:29.051923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:29.052966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:29.052966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:29.052966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:29.052966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:29.052966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:54:29.052966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:54:29.052966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:54:29.052966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:54:29.052966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:54:29.052966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:54:29.052966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:54:29.052966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:54:29.052966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 584/1156] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:54:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:54:29.058245 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61dd89-112e-4285-b46d-5726e8f2a1fc None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:29.083303 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:29.155865 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61dd89-112e-4285-b46d-5726e8f2a1fc None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:29.188054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaedf84-272f-4000-b02c-9638e2e0e213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:29.188669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaedf84-272f-4000-b02c-9638e2e0e213 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:29.188980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaedf84-272f-4000-b02c-9638e2e0e213 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:29.189257 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaedf84-272f-4000-b02c-9638e2e0e213 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:29.189999 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaedf84-272f-4000-b02c-9638e2e0e213 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:29.190309 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaedf84-272f-4000-b02c-9638e2e0e213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:29.192528 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaedf84-272f-4000-b02c-9638e2e0e213 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:29.200487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 573/1157] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:54:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:29.206265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24178b-868a-495f-a1fb-1866d89b7bd8 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:29.310129 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24178b-868a-495f-a1fb-1866d89b7bd8 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:29.343488 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f187f-e23a-4e6b-bac2-1cd3ee201c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:29.344350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f187f-e23a-4e6b-bac2-1cd3ee201c8a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:29.344634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f187f-e23a-4e6b-bac2-1cd3ee201c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:29.344943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f187f-e23a-4e6b-bac2-1cd3ee201c8a None admin] PATH_INFO: `/v3/users/7fe8c3e113134177ac1f6ba224e21b25` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:29.346126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f187f-e23a-4e6b-bac2-1cd3ee201c8a None admin] RBAC: Authorizing `identity:get_user(user_id=7fe8c3e113134177ac1f6ba224e21b25)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:29.347582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f187f-e23a-4e6b-bac2-1cd3ee201c8a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:29.349804 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 585/1158] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:54:29 2022] GET /identity/v3/users/7fe8c3e113134177ac1f6ba224e21b25 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:29.355685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaedf84-272f-4000-b02c-9638e2e0e213 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:29.438979 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:29.453645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaedf84-272f-4000-b02c-9638e2e0e213 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:29.486602 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5de0ec-bd6a-49ca-95a3-63190ade9025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:29.487250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5de0ec-bd6a-49ca-95a3-63190ade9025 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:29.487538 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5de0ec-bd6a-49ca-95a3-63190ade9025 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:29.487817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5de0ec-bd6a-49ca-95a3-63190ade9025 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:29.553176 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5de0ec-bd6a-49ca-95a3-63190ade9025 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:29.554630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5de0ec-bd6a-49ca-95a3-63190ade9025 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:29.638444 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:29.654394 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 574/1159] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:54:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:54:29.660300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f187f-e23a-4e6b-bac2-1cd3ee201c8a None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:29.721244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:29.758905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f187f-e23a-4e6b-bac2-1cd3ee201c8a None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:29.792412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3990d6f-9610-436d-b1e5-df6614a23473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:29.793295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3990d6f-9610-436d-b1e5-df6614a23473 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:29.793574 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3990d6f-9610-436d-b1e5-df6614a23473 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:29.793850 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3990d6f-9610-436d-b1e5-df6614a23473 None admin] PATH_INFO: `/v3/projects/195e7970834244b9bfa0fdffda0548fd` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:29.795017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3990d6f-9610-436d-b1e5-df6614a23473 None admin] RBAC: Authorizing `identity:get_project(project_id=195e7970834244b9bfa0fdffda0548fd)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:29.796459 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3990d6f-9610-436d-b1e5-df6614a23473 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:29.799174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 586/1160] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:54:29 2022] GET /identity/v3/projects/195e7970834244b9bfa0fdffda0548fd => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:29.805532 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5de0ec-bd6a-49ca-95a3-63190ade9025 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:29.901671 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5de0ec-bd6a-49ca-95a3-63190ade9025 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:29.933811 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e3c2a-5c1c-4d53-8261-b62310ac0e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:29.934463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e3c2a-5c1c-4d53-8261-b62310ac0e94 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:29.934742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e3c2a-5c1c-4d53-8261-b62310ac0e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:29.935017 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e3c2a-5c1c-4d53-8261-b62310ac0e94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:29.935838 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e3c2a-5c1c-4d53-8261-b62310ac0e94 None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.project.id=195e7970834244b9bfa0fdffda0548fd, user.id=7fe8c3e113134177ac1f6ba224e21b25) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:29.936175 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e3c2a-5c1c-4d53-8261-b62310ac0e94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:29.937642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e3c2a-5c1c-4d53-8261-b62310ac0e94 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:29.944470 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 575/1161] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:54:29 2022] GET /identity/v3/role_assignments?user.id=7fe8c3e113134177ac1f6ba224e21b25&scope.project.id=195e7970834244b9bfa0fdffda0548fd&role.id=053d1730ed764973abb1490894e07681 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:32.066640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cf184-6542-47ef-8e03-e8a0da9054f1 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:32.067422 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cf184-6542-47ef-8e03-e8a0da9054f1 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:32.067581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20cf184-6542-47ef-8e03-e8a0da9054f1 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:32.068899 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 587/1162] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:32.074301 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3471a-6fe7-4ec4-b5eb-88226ed383b6 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:32.074779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3471a-6fe7-4ec4-b5eb-88226ed383b6 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:32.074856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3471a-6fe7-4ec4-b5eb-88226ed383b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:32.094003 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-17e3471a-6fe7-4ec4-b5eb-88226ed383b6 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:32.095266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-17e3471a-6fe7-4ec4-b5eb-88226ed383b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:32.140791 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:32.161720 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 576/1163] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:32.184253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20cf184-6542-47ef-8e03-e8a0da9054f1 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:32.285202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20cf184-6542-47ef-8e03-e8a0da9054f1 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:32.314017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:32.319610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ca35d-4d15-465e-a400-f334f42f0445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:32.320245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ca35d-4d15-465e-a400-f334f42f0445 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:32.320522 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ca35d-4d15-465e-a400-f334f42f0445 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:32.320824 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ca35d-4d15-465e-a400-f334f42f0445 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:32.388122 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ca35d-4d15-465e-a400-f334f42f0445 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:32.389591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ca35d-4d15-465e-a400-f334f42f0445 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:32.483964 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 588/1164] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:54:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:54:32.490659 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3471a-6fe7-4ec4-b5eb-88226ed383b6 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:32.586079 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3471a-6fe7-4ec4-b5eb-88226ed383b6 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:32.618847 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ebb40-ebbc-48df-98bf-7d7f1909acb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:32.619612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ebb40-ebbc-48df-98bf-7d7f1909acb4 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:32.619890 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ebb40-ebbc-48df-98bf-7d7f1909acb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:32.620161 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ebb40-ebbc-48df-98bf-7d7f1909acb4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:32.626668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ebb40-ebbc-48df-98bf-7d7f1909acb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:32.628102 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ebb40-ebbc-48df-98bf-7d7f1909acb4 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:32.642622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-fc0ebb40-ebbc-48df-98bf-7d7f1909acb4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 12 21:54:32.642622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:32.642622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 12 21:54:32.642622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 12 21:54:32.642622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:54:32.642622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:54:32.642622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:54:32.642622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:54:32.642622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 12 21:54:32.642622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:54:32.642622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:54:32.642622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:54:32.642622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:32.642622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:32.642622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:32.642622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:32.642622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:32.643826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:32.643826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:32.643826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:32.643826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:32.643826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:32.643826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:32.643826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:54:32.643826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:54:32.643826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:54:32.643826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:54:32.643826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:32.643826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:32.643826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:32.643826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:32.643826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:32.643826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:32.644898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:32.644898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:32.644898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:32.644898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:32.644898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:32.644898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:32.644898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:32.644898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:32.644898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:32.644898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:32.644898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 12 21:54:32.644898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:54:32.644898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 12 21:54:32.644898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:54:32.644898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 577/1165] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Jul 12 21:54:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:54:32.650091 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ca35d-4d15-465e-a400-f334f42f0445 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:32.681236 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:32.748637 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ca35d-4d15-465e-a400-f334f42f0445 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:32.781087 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e67d64-03b3-427b-96ea-4098330c4aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:32.781740 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e67d64-03b3-427b-96ea-4098330c4aca None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:32.782025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e67d64-03b3-427b-96ea-4098330c4aca None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:32.782304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e67d64-03b3-427b-96ea-4098330c4aca None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:32.783021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e67d64-03b3-427b-96ea-4098330c4aca None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:32.783333 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e67d64-03b3-427b-96ea-4098330c4aca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:32.784744 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e67d64-03b3-427b-96ea-4098330c4aca None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:32.792346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 589/1166] 10.222.0.44 () {56 vars in 1115 bytes} [Tue Jul 12 21:54:32 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:34.829924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22665f-7e15-4dc5-8764-5bb23f7e422f None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:34.830459 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22665f-7e15-4dc5-8764-5bb23f7e422f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:34.830459 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22665f-7e15-4dc5-8764-5bb23f7e422f None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:34.831844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 578/1167] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:34.837575 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f518c-5ac4-455c-ace8-45873303c8e1 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:34.838001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f518c-5ac4-455c-ace8-45873303c8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:34.838146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f518c-5ac4-455c-ace8-45873303c8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:34.855918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-905f518c-5ac4-455c-ace8-45873303c8e1 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:34.857132 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-905f518c-5ac4-455c-ace8-45873303c8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:34.923690 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 590/1168] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:34.947203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22665f-7e15-4dc5-8764-5bb23f7e422f None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:34.990832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:35.049201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22665f-7e15-4dc5-8764-5bb23f7e422f None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:35.083433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658892b-d2c6-4571-89d8-8fa95f7d0801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:35.084030 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658892b-d2c6-4571-89d8-8fa95f7d0801 None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:35.084315 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658892b-d2c6-4571-89d8-8fa95f7d0801 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:35.084595 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658892b-d2c6-4571-89d8-8fa95f7d0801 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:35.085365 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658892b-d2c6-4571-89d8-8fa95f7d0801 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:35.086748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658892b-d2c6-4571-89d8-8fa95f7d0801 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:35.101714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 579/1169] 10.222.0.44 () {60 vars in 1138 bytes} [Tue Jul 12 21:54:34 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:54:37.167179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9722a-51bc-47d5-bed8-1c159e91d432 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:37.167778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9722a-51bc-47d5-bed8-1c159e91d432 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:37.167778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9722a-51bc-47d5-bed8-1c159e91d432 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:37.169122 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 591/1170] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:37.174232 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309ef08-296f-4b10-82a8-de72ec667438 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:37.174656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309ef08-296f-4b10-82a8-de72ec667438 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:37.174792 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309ef08-296f-4b10-82a8-de72ec667438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:37.192058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-0309ef08-296f-4b10-82a8-de72ec667438 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:37.193475 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0309ef08-296f-4b10-82a8-de72ec667438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:37.257930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 580/1171] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:37.281912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9722a-51bc-47d5-bed8-1c159e91d432 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:37.310579 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:37.386554 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9722a-51bc-47d5-bed8-1c159e91d432 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:37.422696 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425a965-0918-4a6a-b308-eee8335d5b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:37.423559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425a965-0918-4a6a-b308-eee8335d5b27 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:37.423845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425a965-0918-4a6a-b308-eee8335d5b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:37.424126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425a965-0918-4a6a-b308-eee8335d5b27 None admin] PATH_INFO: `/v3/domains/b03703b726bd4cd1a82f38a1c2aa7f8c` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:37.435108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425a965-0918-4a6a-b308-eee8335d5b27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b03703b726bd4cd1a82f38a1c2aa7f8c)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:37.436562 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425a965-0918-4a6a-b308-eee8335d5b27 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:37.438263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 592/1172] 10.222.0.44 () {56 vars in 1183 bytes} [Tue Jul 12 21:54:37 2022] GET /identity/v3/domains/b03703b726bd4cd1a82f38a1c2aa7f8c => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:37.444672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309ef08-296f-4b10-82a8-de72ec667438 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:37.473604 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:37.543561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309ef08-296f-4b10-82a8-de72ec667438 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:37.579488 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182a5ed-eee9-4392-932e-7eff251e4d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:37.580143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182a5ed-eee9-4392-932e-7eff251e4d8e None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:37.580419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182a5ed-eee9-4392-932e-7eff251e4d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:37.580696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182a5ed-eee9-4392-932e-7eff251e4d8e None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:37.581432 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182a5ed-eee9-4392-932e-7eff251e4d8e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:37.583558 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182a5ed-eee9-4392-932e-7eff251e4d8e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:37.608415 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 581/1173] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Jul 12 21:54:37 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:54:39.796214 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966e33e-2b55-4d33-9129-35138481c004 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:39.796901 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966e33e-2b55-4d33-9129-35138481c004 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:39.796901 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966e33e-2b55-4d33-9129-35138481c004 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:39.798162 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 593/1174] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:39.804821 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d0fb2-e3e7-44e7-923d-1b0af0aa96fe None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:39.805340 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d0fb2-e3e7-44e7-923d-1b0af0aa96fe None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:39.805439 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d0fb2-e3e7-44e7-923d-1b0af0aa96fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:39.822749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-d19d0fb2-e3e7-44e7-923d-1b0af0aa96fe None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:39.823955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d19d0fb2-e3e7-44e7-923d-1b0af0aa96fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:39.892143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 582/1175] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:39.917158 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966e33e-2b55-4d33-9129-35138481c004 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:40.007867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:40.024319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966e33e-2b55-4d33-9129-35138481c004 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:40.058929 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870b468-3f61-4db4-a9e7-3184ae40574b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:40.059710 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870b468-3f61-4db4-a9e7-3184ae40574b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:40.059997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870b468-3f61-4db4-a9e7-3184ae40574b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:40.060270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870b468-3f61-4db4-a9e7-3184ae40574b None admin] PATH_INFO: `/v3/domains/b03703b726bd4cd1a82f38a1c2aa7f8c` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:40.061468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870b468-3f61-4db4-a9e7-3184ae40574b None admin] RBAC: Authorizing `identity:get_domain(domain_id=b03703b726bd4cd1a82f38a1c2aa7f8c)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:40.062911 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870b468-3f61-4db4-a9e7-3184ae40574b None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:40.064547 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 594/1176] 10.222.0.44 () {56 vars in 1183 bytes} [Tue Jul 12 21:54:39 2022] GET /identity/v3/domains/b03703b726bd4cd1a82f38a1c2aa7f8c => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:40.071369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d0fb2-e3e7-44e7-923d-1b0af0aa96fe None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:40.087533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:40.178497 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d0fb2-e3e7-44e7-923d-1b0af0aa96fe None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:40.212084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bebd8-cb93-40ef-a113-db2a241047e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:40.212778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bebd8-cb93-40ef-a113-db2a241047e2 None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:40.213066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bebd8-cb93-40ef-a113-db2a241047e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:40.213344 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bebd8-cb93-40ef-a113-db2a241047e2 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:40.214052 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bebd8-cb93-40ef-a113-db2a241047e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:40.215490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bebd8-cb93-40ef-a113-db2a241047e2 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:40.235745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 583/1177] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:54:40 2022] POST /identity/v3/users => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:54:42.294119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb612f6-cb93-45fb-8695-f074c942ef6d None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:42.294704 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb612f6-cb93-45fb-8695-f074c942ef6d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:42.294704 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb612f6-cb93-45fb-8695-f074c942ef6d None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:42.296018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 595/1178] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:42.301167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1086ce5-3f60-43ec-9259-ad8e7e1c1fb7 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:42.301576 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1086ce5-3f60-43ec-9259-ad8e7e1c1fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:42.301733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1086ce5-3f60-43ec-9259-ad8e7e1c1fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:42.318416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-f1086ce5-3f60-43ec-9259-ad8e7e1c1fb7 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:42.319600 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-f1086ce5-3f60-43ec-9259-ad8e7e1c1fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:42.381426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:42.387690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 584/1179] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:42.411580 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb612f6-cb93-45fb-8695-f074c942ef6d None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:42.510858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb612f6-cb93-45fb-8695-f074c942ef6d None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:42.543738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a800f-5284-4d88-b532-c36b98d985b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:42.544564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a800f-5284-4d88-b532-c36b98d985b8 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:42.544833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a800f-5284-4d88-b532-c36b98d985b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:42.545114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a800f-5284-4d88-b532-c36b98d985b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:42.555689 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a800f-5284-4d88-b532-c36b98d985b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:54:42.563244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-200a800f-5284-4d88-b532-c36b98d985b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:54:42.563244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:42.563244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:42.563244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:42.563244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:42.563244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:42.563244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:42.563244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:42.563244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:42.563244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:42.563244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:42.563244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:42.563244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:54:42.563244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:54:42.563244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:54:42.563244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:54:42.564416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:54:42.564416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:54:42.564416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:54:42.564416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:54:42.564416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:42.564416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:42.564416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:42.564416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:42.564416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:42.564416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:42.564416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:42.564416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:42.564416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:42.564416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:42.564416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:42.564416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:42.565821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:42.565821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:42.565821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:42.565821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:42.565821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:54:42.565821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:54:42.565821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:54:42.565821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:54:42.565821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:54:42.565821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:54:42.565821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:54:42.565821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:54:42.565821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 596/1180] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:54:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:54:42.571358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1086ce5-3f60-43ec-9259-ad8e7e1c1fb7 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:42.667169 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1086ce5-3f60-43ec-9259-ad8e7e1c1fb7 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:42.700554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e6b81-fc06-4735-9b76-8b476919cb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:42.701239 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e6b81-fc06-4735-9b76-8b476919cb79 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:42.701521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e6b81-fc06-4735-9b76-8b476919cb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:42.701794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e6b81-fc06-4735-9b76-8b476919cb79 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:42.702528 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e6b81-fc06-4735-9b76-8b476919cb79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:42.702834 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e6b81-fc06-4735-9b76-8b476919cb79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:42.704195 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e6b81-fc06-4735-9b76-8b476919cb79 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:42.714240 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 585/1181] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:54:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:42.720026 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a800f-5284-4d88-b532-c36b98d985b8 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:42.735549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:42.819839 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a800f-5284-4d88-b532-c36b98d985b8 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:42.856905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1916b58-2465-4d49-b71a-77a0d29ddf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:42.857767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1916b58-2465-4d49-b71a-77a0d29ddf32 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:42.858180 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1916b58-2465-4d49-b71a-77a0d29ddf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:42.858468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1916b58-2465-4d49-b71a-77a0d29ddf32 None admin] PATH_INFO: `/v3/users/6bc3a0aebb4a45758f87aae5a6c170f1` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:42.871365 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1916b58-2465-4d49-b71a-77a0d29ddf32 None admin] RBAC: Authorizing `identity:get_user(user_id=6bc3a0aebb4a45758f87aae5a6c170f1)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:42.872855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1916b58-2465-4d49-b71a-77a0d29ddf32 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:42.874518 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 597/1182] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:54:42 2022] GET /identity/v3/users/6bc3a0aebb4a45758f87aae5a6c170f1 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:42.880552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e6b81-fc06-4735-9b76-8b476919cb79 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:42.938851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:42.980820 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e6b81-fc06-4735-9b76-8b476919cb79 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:43.013135 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620cfeed-fe5f-4ab4-ae5b-56f0c2f549ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:43.013779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cfeed-fe5f-4ab4-ae5b-56f0c2f549ad None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:43.014062 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cfeed-fe5f-4ab4-ae5b-56f0c2f549ad None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:43.014340 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cfeed-fe5f-4ab4-ae5b-56f0c2f549ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:43.083685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620cfeed-fe5f-4ab4-ae5b-56f0c2f549ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:43.085079 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620cfeed-fe5f-4ab4-ae5b-56f0c2f549ad None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:43.140671 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:43.181994 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 586/1183] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:54:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:54:43.187947 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1916b58-2465-4d49-b71a-77a0d29ddf32 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:43.258557 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:43.289183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1916b58-2465-4d49-b71a-77a0d29ddf32 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:43.322645 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea42fa5-5f00-4155-baf3-b6d35b7b7b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:43.323482 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea42fa5-5f00-4155-baf3-b6d35b7b7b5a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:43.323759 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea42fa5-5f00-4155-baf3-b6d35b7b7b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:43.324028 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea42fa5-5f00-4155-baf3-b6d35b7b7b5a None admin] PATH_INFO: `/v3/projects/30aa47e833134f5088554342e4260462` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:43.325209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea42fa5-5f00-4155-baf3-b6d35b7b7b5a None admin] RBAC: Authorizing `identity:get_project(project_id=30aa47e833134f5088554342e4260462)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:43.326642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea42fa5-5f00-4155-baf3-b6d35b7b7b5a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:43.328320 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 598/1184] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:54:43 2022] GET /identity/v3/projects/30aa47e833134f5088554342e4260462 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:43.335073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620cfeed-fe5f-4ab4-ae5b-56f0c2f549ad None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:43.437050 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620cfeed-fe5f-4ab4-ae5b-56f0c2f549ad None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:43.471734 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455991b2-9864-4444-8e7a-3e9098cd01ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:43.472385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455991b2-9864-4444-8e7a-3e9098cd01ce None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:43.473234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455991b2-9864-4444-8e7a-3e9098cd01ce None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:43.473596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455991b2-9864-4444-8e7a-3e9098cd01ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:43.474418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455991b2-9864-4444-8e7a-3e9098cd01ce None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.project.id=30aa47e833134f5088554342e4260462, user.id=6bc3a0aebb4a45758f87aae5a6c170f1) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:43.474729 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455991b2-9864-4444-8e7a-3e9098cd01ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:43.476154 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455991b2-9864-4444-8e7a-3e9098cd01ce None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:43.481540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 587/1185] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:54:43 2022] GET /identity/v3/role_assignments?user.id=6bc3a0aebb4a45758f87aae5a6c170f1&scope.project.id=30aa47e833134f5088554342e4260462&role.id=053d1730ed764973abb1490894e07681 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:45.633661 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2368b-a309-4201-acdf-4257f1da0acd None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:45.634213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2368b-a309-4201-acdf-4257f1da0acd None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:45.634213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2368b-a309-4201-acdf-4257f1da0acd None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:45.635590 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 599/1186] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:45.641070 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0c5ea-4834-413e-bacf-2b98d7bf8902 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:45.641593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0c5ea-4834-413e-bacf-2b98d7bf8902 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:45.641593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0c5ea-4834-413e-bacf-2b98d7bf8902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:45.659189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-8ae0c5ea-4834-413e-bacf-2b98d7bf8902 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:45.661165 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8ae0c5ea-4834-413e-bacf-2b98d7bf8902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:45.678976 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:45.732571 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 588/1187] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:45.756059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2368b-a309-4201-acdf-4257f1da0acd None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:45.855835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2368b-a309-4201-acdf-4257f1da0acd None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:45.889096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7a0a4-11b5-4c19-b9a3-0dfc30469a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:45.889824 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7a0a4-11b5-4c19-b9a3-0dfc30469a4d None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:45.890108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7a0a4-11b5-4c19-b9a3-0dfc30469a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:45.890386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7a0a4-11b5-4c19-b9a3-0dfc30469a4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:45.899027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:45.902299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7a0a4-11b5-4c19-b9a3-0dfc30469a4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:54:45.911305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-3bb7a0a4-11b5-4c19-b9a3-0dfc30469a4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:54:45.911305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:45.911305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:45.911305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:45.911305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:45.911305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:45.911305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:45.911305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:45.911305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:45.911305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:45.911305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:45.911305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:45.911305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:54:45.911305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:54:45.911305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:54:45.911305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:54:45.912433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:54:45.912433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:54:45.912433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:54:45.912433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:54:45.912433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:45.912433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:45.912433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:45.912433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:45.912433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:45.912433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:45.912433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:45.912433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:45.912433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:45.912433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:45.912433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:45.912433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:45.913869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:45.913869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:45.913869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:45.913869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:45.913869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:54:45.913869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:54:45.913869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:54:45.913869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:54:45.913869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:54:45.913869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:54:45.913869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:54:45.913869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:54:45.913869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 600/1188] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:54:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:54:45.918665 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0c5ea-4834-413e-bacf-2b98d7bf8902 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:46.018146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0c5ea-4834-413e-bacf-2b98d7bf8902 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:46.053127 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee60538-a364-4f28-b5e1-89d41ddb22cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:46.053795 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee60538-a364-4f28-b5e1-89d41ddb22cc None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:46.054109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee60538-a364-4f28-b5e1-89d41ddb22cc None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:46.054386 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee60538-a364-4f28-b5e1-89d41ddb22cc None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:46.055138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee60538-a364-4f28-b5e1-89d41ddb22cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:46.055450 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee60538-a364-4f28-b5e1-89d41ddb22cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:46.056847 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee60538-a364-4f28-b5e1-89d41ddb22cc None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:46.064876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 589/1189] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:54:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:46.071446 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7a0a4-11b5-4c19-b9a3-0dfc30469a4d None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:46.169014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7a0a4-11b5-4c19-b9a3-0dfc30469a4d None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:46.201861 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddafe41-c4f0-45e3-bfbb-c0dacdacfd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:46.202719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddafe41-c4f0-45e3-bfbb-c0dacdacfd2f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:46.203003 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddafe41-c4f0-45e3-bfbb-c0dacdacfd2f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:46.203281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddafe41-c4f0-45e3-bfbb-c0dacdacfd2f None admin] PATH_INFO: `/v3/users/6bc3a0aebb4a45758f87aae5a6c170f1` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:46.204462 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddafe41-c4f0-45e3-bfbb-c0dacdacfd2f None admin] RBAC: Authorizing `identity:get_user(user_id=6bc3a0aebb4a45758f87aae5a6c170f1)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:46.205943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddafe41-c4f0-45e3-bfbb-c0dacdacfd2f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:46.207794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 601/1190] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:54:46 2022] GET /identity/v3/users/6bc3a0aebb4a45758f87aae5a6c170f1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:46.213612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee60538-a364-4f28-b5e1-89d41ddb22cc None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:46.232259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:46.312671 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee60538-a364-4f28-b5e1-89d41ddb22cc None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:46.345717 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9889a5-80dd-4370-acb9-772a9c4d1e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:46.346567 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9889a5-80dd-4370-acb9-772a9c4d1e34 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:46.346845 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9889a5-80dd-4370-acb9-772a9c4d1e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:46.347118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9889a5-80dd-4370-acb9-772a9c4d1e34 None admin] PATH_INFO: `/v3/projects/30aa47e833134f5088554342e4260462` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:46.348305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9889a5-80dd-4370-acb9-772a9c4d1e34 None admin] RBAC: Authorizing `identity:get_project(project_id=30aa47e833134f5088554342e4260462)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:46.349760 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9889a5-80dd-4370-acb9-772a9c4d1e34 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:46.351442 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 590/1191] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:54:46 2022] GET /identity/v3/projects/30aa47e833134f5088554342e4260462 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:46.357371 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddafe41-c4f0-45e3-bfbb-c0dacdacfd2f None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:46.421701 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:46.454897 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddafe41-c4f0-45e3-bfbb-c0dacdacfd2f None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:46.494629 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd33ba-06b9-4120-aca6-d773fa5235ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:46.495519 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd33ba-06b9-4120-aca6-d773fa5235ee None admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:46.495802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd33ba-06b9-4120-aca6-d773fa5235ee None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:46.496077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd33ba-06b9-4120-aca6-d773fa5235ee None admin] PATH_INFO: `/v3/projects/30aa47e833134f5088554342e4260462/users/6bc3a0aebb4a45758f87aae5a6c170f1/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:46.498060 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd33ba-06b9-4120-aca6-d773fa5235ee None admin] RBAC: Authorizing `identity:create_grant(project_id=30aa47e833134f5088554342e4260462, user_id=6bc3a0aebb4a45758f87aae5a6c170f1, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:46.499495 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd33ba-06b9-4120-aca6-d773fa5235ee None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:46.512945 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 602/1192] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:54:46 2022] PUT /identity/v3/projects/30aa47e833134f5088554342e4260462/users/6bc3a0aebb4a45758f87aae5a6c170f1/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:54:48.551654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f9e12-dc08-45a5-8101-4eae9c2027e6 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:48.552189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f9e12-dc08-45a5-8101-4eae9c2027e6 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:48.552189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f9e12-dc08-45a5-8101-4eae9c2027e6 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:48.553582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 591/1193] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:54:48.559114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14054f7-fef4-4084-ac31-e4ccd5422f6f None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:48.559576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14054f7-fef4-4084-ac31-e4ccd5422f6f None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:48.559670 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14054f7-fef4-4084-ac31-e4ccd5422f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:48.577356 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-c14054f7-fef4-4084-ac31-e4ccd5422f6f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:54:48.578602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c14054f7-fef4-4084-ac31-e4ccd5422f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:54:48.646377 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 603/1194] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:54:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:54:48.669667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f9e12-dc08-45a5-8101-4eae9c2027e6 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:48.767860 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:48.786388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f9e12-dc08-45a5-8101-4eae9c2027e6 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:48.830776 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff7e5c-1e6d-4057-b76b-b028e8c10f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:48.831502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff7e5c-1e6d-4057-b76b-b028e8c10f68 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:48.832217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff7e5c-1e6d-4057-b76b-b028e8c10f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:48.832529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff7e5c-1e6d-4057-b76b-b028e8c10f68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:48.846793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff7e5c-1e6d-4057-b76b-b028e8c10f68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:54:48.855250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-21ff7e5c-1e6d-4057-b76b-b028e8c10f68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:54:48.855250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:54:48.855250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:54:48.855250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:54:48.855250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:54:48.855250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:54:48.855250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:54:48.855250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:54:48.855250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:54:48.855250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:54:48.855250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:54:48.855250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:54:48.855250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:54:48.855250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:54:48.855250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:54:48.855250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:54:48.856388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:54:48.856388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:54:48.856388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:54:48.856388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:54:48.856388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:54:48.856388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:54:48.856388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:54:48.856388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:54:48.856388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:54:48.856388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:54:48.856388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:54:48.856388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:54:48.856388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:54:48.856388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:54:48.856388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:54:48.856388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:54:48.857331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:54:48.857331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:54:48.857331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:54:48.857331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:54:48.857331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:54:48.857331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:54:48.857331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:54:48.857331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:54:48.857331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:54:48.857331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:54:48.857331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:54:48.857331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:54:48.857331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 592/1195] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:54:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:54:48.862691 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14054f7-fef4-4084-ac31-e4ccd5422f6f None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:48.919546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:48.964538 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14054f7-fef4-4084-ac31-e4ccd5422f6f None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:48.997527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de43b93-9f10-44ae-a9b5-4b0cc9b22a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:48.998185 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de43b93-9f10-44ae-a9b5-4b0cc9b22a37 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:48.998468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de43b93-9f10-44ae-a9b5-4b0cc9b22a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:48.998745 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de43b93-9f10-44ae-a9b5-4b0cc9b22a37 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:48.999487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de43b93-9f10-44ae-a9b5-4b0cc9b22a37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:48.999794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de43b93-9f10-44ae-a9b5-4b0cc9b22a37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:49.001201 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de43b93-9f10-44ae-a9b5-4b0cc9b22a37 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:49.009719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 604/1196] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:54:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:49.015755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff7e5c-1e6d-4057-b76b-b028e8c10f68 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:49.116812 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff7e5c-1e6d-4057-b76b-b028e8c10f68 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:49.156822 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d199a-1d7f-4422-9128-3f93ae845b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:49.157748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d199a-1d7f-4422-9128-3f93ae845b2d None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:49.158031 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d199a-1d7f-4422-9128-3f93ae845b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:49.158310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d199a-1d7f-4422-9128-3f93ae845b2d None admin] PATH_INFO: `/v3/users/6bc3a0aebb4a45758f87aae5a6c170f1` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:49.159502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d199a-1d7f-4422-9128-3f93ae845b2d None admin] RBAC: Authorizing `identity:get_user(user_id=6bc3a0aebb4a45758f87aae5a6c170f1)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:49.161635 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d199a-1d7f-4422-9128-3f93ae845b2d None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:49.163585 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 593/1197] 10.222.0.44 () {56 vars in 1177 bytes} [Tue Jul 12 21:54:49 2022] GET /identity/v3/users/6bc3a0aebb4a45758f87aae5a6c170f1 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:49.169131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de43b93-9f10-44ae-a9b5-4b0cc9b22a37 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:49.267063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de43b93-9f10-44ae-a9b5-4b0cc9b22a37 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:49.286313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:49.300301 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04b69c-e5a1-43a4-8782-ae0dfc48b9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:49.301687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04b69c-e5a1-43a4-8782-ae0dfc48b9cb None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:49.301972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04b69c-e5a1-43a4-8782-ae0dfc48b9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:49.302247 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04b69c-e5a1-43a4-8782-ae0dfc48b9cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:49.370798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04b69c-e5a1-43a4-8782-ae0dfc48b9cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:49.372171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04b69c-e5a1-43a4-8782-ae0dfc48b9cb None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:49.478416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 605/1198] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:54:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:54:49.484826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d199a-1d7f-4422-9128-3f93ae845b2d None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:49.502201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:49.587709 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d199a-1d7f-4422-9128-3f93ae845b2d None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:49.624779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587124c-d284-4e5c-a51f-1128fc5e1fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:49.626185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587124c-d284-4e5c-a51f-1128fc5e1fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:49.626471 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587124c-d284-4e5c-a51f-1128fc5e1fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:49.626838 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587124c-d284-4e5c-a51f-1128fc5e1fb7 None admin] PATH_INFO: `/v3/projects/30aa47e833134f5088554342e4260462` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:49.628185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587124c-d284-4e5c-a51f-1128fc5e1fb7 None admin] RBAC: Authorizing `identity:get_project(project_id=30aa47e833134f5088554342e4260462)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:49.630777 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587124c-d284-4e5c-a51f-1128fc5e1fb7 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:49.632500 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 594/1199] 10.222.0.44 () {56 vars in 1186 bytes} [Tue Jul 12 21:54:49 2022] GET /identity/v3/projects/30aa47e833134f5088554342e4260462 => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:49.638621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04b69c-e5a1-43a4-8782-ae0dfc48b9cb None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:54:49.659069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:49.734092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04b69c-e5a1-43a4-8782-ae0dfc48b9cb None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:54:49.768275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed238f-aa40-4994-8041-15fedb993a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:54:49.768945 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed238f-aa40-4994-8041-15fedb993a4a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:54:49.769255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed238f-aa40-4994-8041-15fedb993a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:54:49.769531 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed238f-aa40-4994-8041-15fedb993a4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:54:49.770351 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed238f-aa40-4994-8041-15fedb993a4a None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.project.id=30aa47e833134f5088554342e4260462, user.id=6bc3a0aebb4a45758f87aae5a6c170f1) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:54:49.770675 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed238f-aa40-4994-8041-15fedb993a4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:54:49.772079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed238f-aa40-4994-8041-15fedb993a4a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:54:49.777796 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 606/1200] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:54:49 2022] GET /identity/v3/role_assignments?user.id=6bc3a0aebb4a45758f87aae5a6c170f1&scope.project.id=30aa47e833134f5088554342e4260462&role.id=053d1730ed764973abb1490894e07681 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:54:59.617485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=60 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:54:59.775624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=58 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:45.940499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d8947-c946-4914-abdd-90c83cdc4205 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:45.941419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d8947-c946-4914-abdd-90c83cdc4205 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:45.941419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d8947-c946-4914-abdd-90c83cdc4205 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:45.942850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 595/1201] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:55:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:55:45.948621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da100d8-229e-477f-b4b2-fb546b6ea5cc None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:45.949168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da100d8-229e-477f-b4b2-fb546b6ea5cc None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:45.949254 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da100d8-229e-477f-b4b2-fb546b6ea5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:45.975466 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-2da100d8-229e-477f-b4b2-fb546b6ea5cc None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:55:45.976840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2da100d8-229e-477f-b4b2-fb546b6ea5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:55:46.057949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 607/1202] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:55:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:55:46.090745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d8947-c946-4914-abdd-90c83cdc4205 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:46.232475 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d8947-c946-4914-abdd-90c83cdc4205 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:46.277384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c5a62-6a57-425e-a807-a9846312d848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:46.278371 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c5a62-6a57-425e-a807-a9846312d848 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:46.278855 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c5a62-6a57-425e-a807-a9846312d848 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:46.279349 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c5a62-6a57-425e-a807-a9846312d848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:46.293090 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c5a62-6a57-425e-a807-a9846312d848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:46.294691 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c5a62-6a57-425e-a807-a9846312d848 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:46.320426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-cf8c5a62-6a57-425e-a807-a9846312d848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:55:46.320426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:55:46.320426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:55:46.320426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:55:46.320426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:55:46.320426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:55:46.320426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:55:46.320426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:55:46.320426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:55:46.320426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:55:46.320426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:55:46.320426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:55:46.320426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:55:46.320426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:55:46.320426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:55:46.321988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:55:46.321988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:55:46.321988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:55:46.321988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:55:46.321988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:55:46.321988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:55:46.321988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:55:46.321988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:55:46.321988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:55:46.321988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:55:46.321988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:55:46.321988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:55:46.321988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:55:46.321988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:55:46.321988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:55:46.321988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:55:46.321988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:55:46.323206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:55:46.323206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:55:46.323206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:55:46.323206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:55:46.325666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 596/1203] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:55:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:55:46.328842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da100d8-229e-477f-b4b2-fb546b6ea5cc None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:46.479330 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da100d8-229e-477f-b4b2-fb546b6ea5cc None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:46.522421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:46.532749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1db2f-ff66-4121-bf35-3fc7b16b7942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:46.533705 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1db2f-ff66-4121-bf35-3fc7b16b7942 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:46.534032 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1db2f-ff66-4121-bf35-3fc7b16b7942 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:46.534601 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1db2f-ff66-4121-bf35-3fc7b16b7942 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:46.535570 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1db2f-ff66-4121-bf35-3fc7b16b7942 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:55:46.535950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1db2f-ff66-4121-bf35-3fc7b16b7942 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:46.540064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1db2f-ff66-4121-bf35-3fc7b16b7942 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:46.555173 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 608/1204] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:55:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:55:46.561306 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c5a62-6a57-425e-a807-a9846312d848 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:46.630048 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:46.704546 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c5a62-6a57-425e-a807-a9846312d848 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:46.754131 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c3049-ed2b-42fb-8b8a-3d2f510e4043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:46.754875 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c3049-ed2b-42fb-8b8a-3d2f510e4043 None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:46.755190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c3049-ed2b-42fb-8b8a-3d2f510e4043 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:46.755478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c3049-ed2b-42fb-8b8a-3d2f510e4043 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:46.756256 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c3049-ed2b-42fb-8b8a-3d2f510e4043 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:46.757957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c3049-ed2b-42fb-8b8a-3d2f510e4043 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:46.782963 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 597/1205] 10.222.0.44 () {60 vars in 1132 bytes} [Tue Jul 12 21:55:46 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 21:55:49.228276 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865c8f9-b336-4b78-8658-5c1011b93316 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:49.228932 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865c8f9-b336-4b78-8658-5c1011b93316 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:49.228932 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865c8f9-b336-4b78-8658-5c1011b93316 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:49.230272 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 609/1206] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:55:49.236376 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60f6ea-bfe9-47f8-99eb-d62b69d75b88 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:49.236937 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60f6ea-bfe9-47f8-99eb-d62b69d75b88 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:49.236937 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60f6ea-bfe9-47f8-99eb-d62b69d75b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:49.256765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-de60f6ea-bfe9-47f8-99eb-d62b69d75b88 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:55:49.258214 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-de60f6ea-bfe9-47f8-99eb-d62b69d75b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:55:49.341388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 598/1207] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:55:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:55:49.366263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865c8f9-b336-4b78-8658-5c1011b93316 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:49.503129 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865c8f9-b336-4b78-8658-5c1011b93316 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:49.545210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ecd08-1f1f-4cd8-895e-647fc3024f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:49.548133 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ecd08-1f1f-4cd8-895e-647fc3024f13 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:49.548622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ecd08-1f1f-4cd8-895e-647fc3024f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:49.548926 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ecd08-1f1f-4cd8-895e-647fc3024f13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:49.565373 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ecd08-1f1f-4cd8-895e-647fc3024f13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:55:49.575868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-2d7ecd08-1f1f-4cd8-895e-647fc3024f13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:55:49.575868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:55:49.575868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:55:49.575868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:55:49.575868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:55:49.575868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:55:49.575868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:55:49.575868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:55:49.575868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:55:49.575868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:55:49.575868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:55:49.575868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:55:49.575868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:55:49.575868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:55:49.575868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:55:49.575868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:55:49.577146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:55:49.577146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:55:49.577146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:55:49.577146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:55:49.577146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:55:49.577146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:55:49.577146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:55:49.577146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:55:49.577146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:55:49.577146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:55:49.577146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:55:49.577146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:55:49.577146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:55:49.577146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:55:49.577146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:55:49.577146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:55:49.578157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:55:49.578157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:55:49.578157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:55:49.578157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:55:49.578157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:55:49.578157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:55:49.578157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:55:49.578157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:55:49.578157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:55:49.578157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:55:49.578157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:55:49.578157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:55:49.578157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 610/1208] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:55:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:55:49.584794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60f6ea-bfe9-47f8-99eb-d62b69d75b88 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:49.640110 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:49.716600 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60f6ea-bfe9-47f8-99eb-d62b69d75b88 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:49.761743 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090baf76-4902-4598-9625-1c1e494d9b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:49.762479 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090baf76-4902-4598-9625-1c1e494d9b78 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:49.762895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090baf76-4902-4598-9625-1c1e494d9b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:49.763181 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090baf76-4902-4598-9625-1c1e494d9b78 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:49.763944 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090baf76-4902-4598-9625-1c1e494d9b78 None admin] RBAC: Adding query filter params (name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:55:49.764259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090baf76-4902-4598-9625-1c1e494d9b78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:49.765661 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090baf76-4902-4598-9625-1c1e494d9b78 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:49.775363 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 599/1209] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:55:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:55:49.782025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ecd08-1f1f-4cd8-895e-647fc3024f13 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:49.840634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:49.903865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ecd08-1f1f-4cd8-895e-647fc3024f13 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:49.950035 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c18eb6-af4d-4d13-9ef5-9dda248ed72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:49.951095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18eb6-af4d-4d13-9ef5-9dda248ed72a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:49.951385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18eb6-af4d-4d13-9ef5-9dda248ed72a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:49.951673 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18eb6-af4d-4d13-9ef5-9dda248ed72a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:49.965538 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c18eb6-af4d-4d13-9ef5-9dda248ed72a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:49.967151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c18eb6-af4d-4d13-9ef5-9dda248ed72a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:49.994838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-a6c18eb6-af4d-4d13-9ef5-9dda248ed72a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:55:49.994838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:55:49.994838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:55:49.994838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:55:49.994838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:55:49.994838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:55:49.994838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:55:49.994838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:55:49.994838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:55:49.994838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:55:49.994838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:55:49.994838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:55:49.994838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:55:49.994838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:55:49.994838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:55:49.995949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:55:49.995949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:55:49.995949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:55:49.995949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:55:49.995949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:55:49.995949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:55:49.995949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:55:49.995949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:55:49.995949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:55:49.995949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:55:49.995949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:55:49.995949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:55:49.995949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:55:49.995949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:55:49.995949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:55:49.995949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:55:49.995949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:55:49.996987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:55:49.996987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:55:49.996987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:55:49.996987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:55:49.996987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 611/1210] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:55:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:55:50.002370 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090baf76-4902-4598-9625-1c1e494d9b78 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:50.127371 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:50.131011 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090baf76-4902-4598-9625-1c1e494d9b78 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:50.175004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dacb7f-2c82-4543-a1a0-f21fbbda8cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:50.175987 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dacb7f-2c82-4543-a1a0-f21fbbda8cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:50.176414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dacb7f-2c82-4543-a1a0-f21fbbda8cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:50.176891 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dacb7f-2c82-4543-a1a0-f21fbbda8cb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:50.177950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dacb7f-2c82-4543-a1a0-f21fbbda8cb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:55:50.178273 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dacb7f-2c82-4543-a1a0-f21fbbda8cb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:50.179909 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dacb7f-2c82-4543-a1a0-f21fbbda8cb5 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:50.196148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 600/1211] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:55:50.201431 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c18eb6-af4d-4d13-9ef5-9dda248ed72a None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:50.323496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c18eb6-af4d-4d13-9ef5-9dda248ed72a None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:50.325278 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:50.367783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c36f376-e942-4417-be19-f43c13372583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:50.368718 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36f376-e942-4417-be19-f43c13372583 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:50.369079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36f376-e942-4417-be19-f43c13372583 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:50.369418 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36f376-e942-4417-be19-f43c13372583 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:50.379472 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c36f376-e942-4417-be19-f43c13372583 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:50.380979 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c36f376-e942-4417-be19-f43c13372583 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:50.399229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-3c36f376-e942-4417-be19-f43c13372583 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 21:55:50.399229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:55:50.399229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:55:50.399229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:55:50.399229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:55:50.399229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:55:50.399229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:55:50.399229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:55:50.399229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:55:50.399229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:55:50.399229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:55:50.399229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:55:50.399229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:55:50.399229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:55:50.399229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:55:50.399229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:55:50.400244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:55:50.401287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:55:50.401287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:55:50.401287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:55:50.401287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:55:50.401287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:55:50.401287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:55:50.401287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:55:50.401287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:55:50.401287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 21:55:50.401287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:55:50.401287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 612/1212] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:55:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:55:50.407024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dacb7f-2c82-4543-a1a0-f21fbbda8cb5 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:50.531036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dacb7f-2c82-4543-a1a0-f21fbbda8cb5 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:50.582882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dcd55-c7c7-42c3-9513-5efdfaf6e498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:50.584482 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dcd55-c7c7-42c3-9513-5efdfaf6e498 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:50.584482 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dcd55-c7c7-42c3-9513-5efdfaf6e498 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:50.584482 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dcd55-c7c7-42c3-9513-5efdfaf6e498 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:50.585042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1dcd55-c7c7-42c3-9513-5efdfaf6e498 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:55:50.585351 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1dcd55-c7c7-42c3-9513-5efdfaf6e498 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:50.586999 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1dcd55-c7c7-42c3-9513-5efdfaf6e498 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:50.606582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 601/1213] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Jul 12 21:55:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:55:50.613282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c36f376-e942-4417-be19-f43c13372583 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:50.761718 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c36f376-e942-4417-be19-f43c13372583 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:50.834471 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85ee92-c0b9-4d3b-ada4-8099e5b1567c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:50.835663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85ee92-c0b9-4d3b-ada4-8099e5b1567c None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:50.835663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85ee92-c0b9-4d3b-ada4-8099e5b1567c None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:50.835870 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85ee92-c0b9-4d3b-ada4-8099e5b1567c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:50.882073 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:50.932835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85ee92-c0b9-4d3b-ada4-8099e5b1567c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:50.934305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85ee92-c0b9-4d3b-ada4-8099e5b1567c None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:51.067347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 613/1214] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:55:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:55:51.068778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dcd55-c7c7-42c3-9513-5efdfaf6e498 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:51.089357 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:51.182502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dcd55-c7c7-42c3-9513-5efdfaf6e498 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:51.222463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7757bc-7e0c-4db6-bc41-2c3cc83e2bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:51.223265 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7757bc-7e0c-4db6-bc41-2c3cc83e2bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:51.223549 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7757bc-7e0c-4db6-bc41-2c3cc83e2bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:51.223824 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7757bc-7e0c-4db6-bc41-2c3cc83e2bb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:51.234554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7757bc-7e0c-4db6-bc41-2c3cc83e2bb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:51.236813 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7757bc-7e0c-4db6-bc41-2c3cc83e2bb4 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:51.256474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-0f7757bc-7e0c-4db6-bc41-2c3cc83e2bb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:55:51.256474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:55:51.256474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:55:51.256474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:55:51.256474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:55:51.256474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:55:51.256474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:55:51.256474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:55:51.256474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:55:51.256474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:55:51.256474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:55:51.256474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:55:51.256474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:55:51.256474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:55:51.256474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:55:51.257509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:55:51.257509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:55:51.257509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:55:51.257509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:55:51.257509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:55:51.257509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:55:51.257509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:55:51.257509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:55:51.257509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:55:51.257509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:55:51.257509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:55:51.257509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:55:51.257509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:55:51.257509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:55:51.257509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:55:51.257509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:55:51.257509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:55:51.258455 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:55:51.258455 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:55:51.258455 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:55:51.258455 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:55:51.258455 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 602/1215] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:55:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:55:51.264637 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85ee92-c0b9-4d3b-ada4-8099e5b1567c None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:51.327934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:51.368762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85ee92-c0b9-4d3b-ada4-8099e5b1567c None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:51.409566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5f8a7-d2e9-4e42-b6d0-319f49cbd661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:51.410235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5f8a7-d2e9-4e42-b6d0-319f49cbd661 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:51.410518 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5f8a7-d2e9-4e42-b6d0-319f49cbd661 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:51.410789 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5f8a7-d2e9-4e42-b6d0-319f49cbd661 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:51.411514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5f8a7-d2e9-4e42-b6d0-319f49cbd661 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:55:51.411827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5f8a7-d2e9-4e42-b6d0-319f49cbd661 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:51.413230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5f8a7-d2e9-4e42-b6d0-319f49cbd661 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:51.425778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 614/1216] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:55:51.431333 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7757bc-7e0c-4db6-bc41-2c3cc83e2bb4 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:51.514214 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:51.557101 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7757bc-7e0c-4db6-bc41-2c3cc83e2bb4 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:51.599091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf4d35-e572-49d5-9725-d21a676ea9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:51.599788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf4d35-e572-49d5-9725-d21a676ea9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:51.600071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf4d35-e572-49d5-9725-d21a676ea9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:51.600348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf4d35-e572-49d5-9725-d21a676ea9f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:51.607723 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf4d35-e572-49d5-9725-d21a676ea9f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:51.609326 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf4d35-e572-49d5-9725-d21a676ea9f2 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:51.625806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-2fbf4d35-e572-49d5-9725-d21a676ea9f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:55:51.625806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:55:51.625806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:55:51.625806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:55:51.625806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:55:51.625806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:55:51.625806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:55:51.625806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:55:51.625806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:55:51.625806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:55:51.625806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:55:51.625806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:55:51.625806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:55:51.625806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:55:51.625806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:55:51.627150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:55:51.627150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:55:51.627150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:55:51.627150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:55:51.627150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:55:51.627150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:55:51.627150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:55:51.627150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:55:51.627150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:55:51.627150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:55:51.627150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:55:51.627150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:55:51.627150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:55:51.627150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:55:51.627150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:55:51.627150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:55:51.627150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:55:51.628071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:55:51.628071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:55:51.628071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:55:51.628071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:55:51.628071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:55:51.628071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:55:51.628071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:55:51.628071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:55:51.628071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 603/1217] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:55:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:55:51.633376 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5f8a7-d2e9-4e42-b6d0-319f49cbd661 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:51.756912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5f8a7-d2e9-4e42-b6d0-319f49cbd661 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:51.781745 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:51.797128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfedfae-4c03-47f5-b08e-0e4d532e5634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:51.798014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfedfae-4c03-47f5-b08e-0e4d532e5634 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:51.798395 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfedfae-4c03-47f5-b08e-0e4d532e5634 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:51.798711 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfedfae-4c03-47f5-b08e-0e4d532e5634 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:51.799582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfedfae-4c03-47f5-b08e-0e4d532e5634 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:55:51.799898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfedfae-4c03-47f5-b08e-0e4d532e5634 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:51.801620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfedfae-4c03-47f5-b08e-0e4d532e5634 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:51.816329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 615/1218] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:55:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:55:51.826036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf4d35-e572-49d5-9725-d21a676ea9f2 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:51.966959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf4d35-e572-49d5-9725-d21a676ea9f2 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:52.002713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:52.014619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a47df6-561b-4d94-b052-991446d24b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:52.015309 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a47df6-561b-4d94-b052-991446d24b63 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:52.015599 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a47df6-561b-4d94-b052-991446d24b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:52.015880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a47df6-561b-4d94-b052-991446d24b63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:52.016821 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a47df6-561b-4d94-b052-991446d24b63 None admin] RBAC: Adding query filter params (role.id=f1e42ea5a1c440bba915b459d2744858, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=a1f1ef52f8eb42a59a50a69873b9ffaa) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:55:52.017201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a47df6-561b-4d94-b052-991446d24b63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:52.018635 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a47df6-561b-4d94-b052-991446d24b63 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:52.029185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 604/1219] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:55:51 2022] GET /identity/v3/role_assignments?user.id=a1f1ef52f8eb42a59a50a69873b9ffaa&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=f1e42ea5a1c440bba915b459d2744858 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:55:54.170017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed2ad2-92d0-4eba-a361-d7cdea054404 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:54.170594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed2ad2-92d0-4eba-a361-d7cdea054404 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:54.170982 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed2ad2-92d0-4eba-a361-d7cdea054404 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:54.173119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 616/1220] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:55:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:55:54.178975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249b8cc-2675-47d4-9447-9cbcf3d0e0c5 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:54.179478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249b8cc-2675-47d4-9447-9cbcf3d0e0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:54.179583 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249b8cc-2675-47d4-9447-9cbcf3d0e0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:54.197107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-c249b8cc-2675-47d4-9447-9cbcf3d0e0c5 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:55:54.198423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c249b8cc-2675-47d4-9447-9cbcf3d0e0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:55:54.274461 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 605/1221] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:55:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:55:54.296930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed2ad2-92d0-4eba-a361-d7cdea054404 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:54.414315 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed2ad2-92d0-4eba-a361-d7cdea054404 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:54.450869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19f155-8398-458c-a8b6-aa57a918b03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:54.451657 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19f155-8398-458c-a8b6-aa57a918b03b None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:54.451934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19f155-8398-458c-a8b6-aa57a918b03b None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:54.452211 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19f155-8398-458c-a8b6-aa57a918b03b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:54.465570 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19f155-8398-458c-a8b6-aa57a918b03b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:55:54.474166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-7c19f155-8398-458c-a8b6-aa57a918b03b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:55:54.474166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:55:54.474166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:55:54.474166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:55:54.474166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:55:54.474166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:55:54.474166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:55:54.474166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:55:54.474166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:55:54.474166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:55:54.474166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:55:54.474166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:55:54.474166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:55:54.474166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:55:54.474166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:55:54.474166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:55:54.475285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:55:54.475285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:55:54.475285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:55:54.475285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:55:54.475285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:55:54.475285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:55:54.475285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:55:54.475285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:55:54.475285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:55:54.475285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:55:54.475285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:55:54.475285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:55:54.475285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:55:54.475285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:55:54.475285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:55:54.475285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:55:54.476441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:55:54.476441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:55:54.476441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:55:54.476441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:55:54.476441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:55:54.476441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:55:54.476441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:55:54.476441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:55:54.476441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:55:54.476441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:55:54.476441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:55:54.476441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:55:54.476441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 617/1222] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:55:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:55:54.481572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249b8cc-2675-47d4-9447-9cbcf3d0e0c5 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:54.593404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249b8cc-2675-47d4-9447-9cbcf3d0e0c5 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:54.622320 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:54.634621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6ba8a-72a7-4acb-8a33-58d19823049f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:54.635289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6ba8a-72a7-4acb-8a33-58d19823049f None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:54.635572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6ba8a-72a7-4acb-8a33-58d19823049f None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:54.635851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6ba8a-72a7-4acb-8a33-58d19823049f None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:54.636599 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6ba8a-72a7-4acb-8a33-58d19823049f None admin] RBAC: Adding query filter params (name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:55:54.636938 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6ba8a-72a7-4acb-8a33-58d19823049f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:54.638313 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6ba8a-72a7-4acb-8a33-58d19823049f None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:54.649384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 606/1223] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:55:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:55:54.655831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19f155-8398-458c-a8b6-aa57a918b03b None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:54.706075 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:54.767949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19f155-8398-458c-a8b6-aa57a918b03b None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:54.805460 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab123dae-14b8-4182-bdef-c55d77d09417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:54.806266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab123dae-14b8-4182-bdef-c55d77d09417 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:54.806553 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab123dae-14b8-4182-bdef-c55d77d09417 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:54.806827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab123dae-14b8-4182-bdef-c55d77d09417 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:54.818800 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab123dae-14b8-4182-bdef-c55d77d09417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:54.820241 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab123dae-14b8-4182-bdef-c55d77d09417 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:54.841548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-ab123dae-14b8-4182-bdef-c55d77d09417 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:55:54.841548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:55:54.841548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:55:54.841548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:55:54.841548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:55:54.841548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:55:54.841548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:55:54.841548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:55:54.841548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:55:54.841548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:55:54.841548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:55:54.841548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:55:54.841548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:55:54.841548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:55:54.841548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:55:54.842647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:55:54.842647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:55:54.842647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:55:54.842647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:55:54.842647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:55:54.842647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:55:54.842647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:55:54.842647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:55:54.842647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:55:54.842647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:55:54.842647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:55:54.842647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:55:54.842647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:55:54.842647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:55:54.842647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:55:54.842647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:55:54.842647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:55:54.843727 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:55:54.843727 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:55:54.843727 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:55:54.843727 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:55:54.843727 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 618/1224] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:55:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:55:54.849692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6ba8a-72a7-4acb-8a33-58d19823049f None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:54.960039 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6ba8a-72a7-4acb-8a33-58d19823049f None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:54.999675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9f5e6-c87e-448f-9546-5d75591c4da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:55.000412 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9f5e6-c87e-448f-9546-5d75591c4da4 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:55.000910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9f5e6-c87e-448f-9546-5d75591c4da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:55.001225 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9f5e6-c87e-448f-9546-5d75591c4da4 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:55.001929 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9f5e6-c87e-448f-9546-5d75591c4da4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:55:55.002243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9f5e6-c87e-448f-9546-5d75591c4da4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:55.003623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9f5e6-c87e-448f-9546-5d75591c4da4 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:55.021623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 607/1225] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:55:55.027983 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab123dae-14b8-4182-bdef-c55d77d09417 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:55.130431 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:55.147364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab123dae-14b8-4182-bdef-c55d77d09417 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:55.189632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cba99-6f56-40e9-89cd-7c594c38062f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:55.190439 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cba99-6f56-40e9-89cd-7c594c38062f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:55.191034 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cba99-6f56-40e9-89cd-7c594c38062f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:55.191290 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cba99-6f56-40e9-89cd-7c594c38062f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:55.201072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6cba99-6f56-40e9-89cd-7c594c38062f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:55.202640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6cba99-6f56-40e9-89cd-7c594c38062f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:55.219209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-8a6cba99-6f56-40e9-89cd-7c594c38062f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 21:55:55.219209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:55:55.219209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:55:55.219209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:55:55.219209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:55:55.219209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:55:55.219209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:55:55.219209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:55:55.219209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:55:55.219209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:55:55.219209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:55:55.219209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:55:55.219209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:55:55.219209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:55:55.219209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:55:55.219209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:55:55.220430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:55:55.221735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:55:55.221735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:55:55.221735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:55:55.221735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:55:55.221735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:55:55.221735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:55:55.221735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:55:55.221735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:55:55.221735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 21:55:55.221735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:55:55.221735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 619/1226] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:55:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 21:55:55.226886 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9f5e6-c87e-448f-9546-5d75591c4da4 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:55.284018 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:55.346901 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9f5e6-c87e-448f-9546-5d75591c4da4 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:55.386872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3edbb-5405-4080-90d8-ca271c8015a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:55.387480 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3edbb-5405-4080-90d8-ca271c8015a2 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:55.387765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3edbb-5405-4080-90d8-ca271c8015a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:55.388043 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3edbb-5405-4080-90d8-ca271c8015a2 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:55.389160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3edbb-5405-4080-90d8-ca271c8015a2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:55:55.389467 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3edbb-5405-4080-90d8-ca271c8015a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:55.391304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3edbb-5405-4080-90d8-ca271c8015a2 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:55.411575 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 608/1227] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Jul 12 21:55:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:55:55.417293 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cba99-6f56-40e9-89cd-7c594c38062f None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:55.540917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cba99-6f56-40e9-89cd-7c594c38062f None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:55.584354 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980647e2-d1f3-4700-bde1-9b6c0fee857c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:55.585357 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980647e2-d1f3-4700-bde1-9b6c0fee857c None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:55.585846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980647e2-d1f3-4700-bde1-9b6c0fee857c None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:55.586366 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980647e2-d1f3-4700-bde1-9b6c0fee857c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:55.606583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980647e2-d1f3-4700-bde1-9b6c0fee857c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:55.609775 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980647e2-d1f3-4700-bde1-9b6c0fee857c None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:55.635665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-980647e2-d1f3-4700-bde1-9b6c0fee857c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:55:55.635665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:55:55.635665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:55:55.635665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:55:55.635665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:55:55.635665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:55:55.635665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:55:55.635665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:55:55.635665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:55:55.635665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:55:55.635665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:55:55.635665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:55:55.635665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:55:55.635665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:55:55.635665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:55:55.636635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:55:55.636635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:55:55.636635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:55:55.636635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:55:55.636635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:55:55.636635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:55:55.636635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:55:55.636635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:55:55.636635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:55:55.636635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:55:55.636635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:55:55.636635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:55:55.636635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:55:55.636635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:55:55.636635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:55:55.636635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:55:55.636635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:55:55.637636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:55:55.637636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:55:55.637636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:55:55.637636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:55:55.637636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 620/1228] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:55:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:55:55.643958 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3edbb-5405-4080-90d8-ca271c8015a2 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:55.763256 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3edbb-5405-4080-90d8-ca271c8015a2 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:55.768322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:55.805061 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc01ee-9453-44b2-97f5-b66324036958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:55.805823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc01ee-9453-44b2-97f5-b66324036958 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:55.806146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc01ee-9453-44b2-97f5-b66324036958 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:55.806562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc01ee-9453-44b2-97f5-b66324036958 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:55.807338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc01ee-9453-44b2-97f5-b66324036958 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:55:55.807783 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc01ee-9453-44b2-97f5-b66324036958 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:55.809479 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc01ee-9453-44b2-97f5-b66324036958 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:55.824876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 609/1229] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:55:55.833888 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980647e2-d1f3-4700-bde1-9b6c0fee857c None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:55.843810 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:55.956267 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980647e2-d1f3-4700-bde1-9b6c0fee857c None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:55.995935 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d202f3-68db-4390-8be5-61b898007101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:55.996809 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d202f3-68db-4390-8be5-61b898007101 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:55.997095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d202f3-68db-4390-8be5-61b898007101 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:55.997410 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d202f3-68db-4390-8be5-61b898007101 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:56.007103 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d202f3-68db-4390-8be5-61b898007101 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:56.008911 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d202f3-68db-4390-8be5-61b898007101 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:56.024872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-a0d202f3-68db-4390-8be5-61b898007101 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:55:56.024872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:55:56.024872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:55:56.024872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:55:56.024872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:55:56.024872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:55:56.024872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:55:56.024872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:55:56.024872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:55:56.024872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:55:56.024872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:55:56.024872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:55:56.024872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:55:56.024872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:55:56.024872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:55:56.025841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:55:56.025841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:55:56.025841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:55:56.025841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:55:56.025841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:55:56.025841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:55:56.025841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:55:56.025841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:55:56.025841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:55:56.025841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:55:56.025841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:55:56.025841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:55:56.025841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:55:56.025841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:55:56.025841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:55:56.025841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:55:56.025841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:55:56.026992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:55:56.026992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:55:56.026992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:55:56.026992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:55:56.026992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:55:56.026992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:55:56.026992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:55:56.026992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:55:56.026992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 621/1230] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:55:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:55:56.032338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc01ee-9453-44b2-97f5-b66324036958 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:56.151235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc01ee-9453-44b2-97f5-b66324036958 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:56.191347 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9206d-afd0-4d70-817e-52828ba0c215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:56.192033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9206d-afd0-4d70-817e-52828ba0c215 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:56.192316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9206d-afd0-4d70-817e-52828ba0c215 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:56.192588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9206d-afd0-4d70-817e-52828ba0c215 None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:56.193418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9206d-afd0-4d70-817e-52828ba0c215 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:55:56.193724 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9206d-afd0-4d70-817e-52828ba0c215 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:56.195160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9206d-afd0-4d70-817e-52828ba0c215 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:56.207892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 610/1231] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:55:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:55:56.214013 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d202f3-68db-4390-8be5-61b898007101 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:56.287369 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:56.333645 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d202f3-68db-4390-8be5-61b898007101 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:56.376757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3b86d-c38a-40b5-acf0-44f1e1cb266f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:56.377695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3b86d-c38a-40b5-acf0-44f1e1cb266f None admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:56.378023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3b86d-c38a-40b5-acf0-44f1e1cb266f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:56.378367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3b86d-c38a-40b5-acf0-44f1e1cb266f None admin] PATH_INFO: `/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/a1f1ef52f8eb42a59a50a69873b9ffaa/roles/f1e42ea5a1c440bba915b459d2744858` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:56.396735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3b86d-c38a-40b5-acf0-44f1e1cb266f None admin] RBAC: Authorizing `identity:create_grant(project_id=459e4743ff3043f69408ca2095be3bd0, user_id=a1f1ef52f8eb42a59a50a69873b9ffaa, role_id=f1e42ea5a1c440bba915b459d2744858)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:56.398362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3b86d-c38a-40b5-acf0-44f1e1cb266f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:56.413232 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 622/1232] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:55:56 2022] PUT /identity/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/a1f1ef52f8eb42a59a50a69873b9ffaa/roles/f1e42ea5a1c440bba915b459d2744858 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:55:58.702343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494902c-26fa-45b7-93d1-d9ba954e4192 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:58.702919 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494902c-26fa-45b7-93d1-d9ba954e4192 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:58.702919 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494902c-26fa-45b7-93d1-d9ba954e4192 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:58.704883 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 611/1233] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:55:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:55:58.711921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40cde9-163a-4dea-915c-6a79f514a3f7 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:58.712651 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40cde9-163a-4dea-915c-6a79f514a3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:58.712905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40cde9-163a-4dea-915c-6a79f514a3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:58.735999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-4f40cde9-163a-4dea-915c-6a79f514a3f7 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:55:58.737813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-4f40cde9-163a-4dea-915c-6a79f514a3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:55:58.816882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 623/1234] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:55:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 21:55:58.847230 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494902c-26fa-45b7-93d1-d9ba954e4192 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:58.884584 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:58.980995 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494902c-26fa-45b7-93d1-d9ba954e4192 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:59.023345 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214b56f-5fbb-4b25-a024-72ab0912f401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:59.024169 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214b56f-5fbb-4b25-a024-72ab0912f401 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:59.024449 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214b56f-5fbb-4b25-a024-72ab0912f401 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:59.024734 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214b56f-5fbb-4b25-a024-72ab0912f401 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:59.035654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214b56f-5fbb-4b25-a024-72ab0912f401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:55:59.044381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-4214b56f-5fbb-4b25-a024-72ab0912f401 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:55:59.044381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:55:59.044381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:55:59.044381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:55:59.044381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:55:59.044381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:55:59.044381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:55:59.044381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:55:59.044381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:55:59.044381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:55:59.044381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:55:59.044381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:55:59.044381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:55:59.044381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:55:59.044381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:55:59.044381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:55:59.045458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:55:59.045458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:55:59.045458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:55:59.045458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:55:59.045458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:55:59.045458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:55:59.045458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:55:59.045458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:55:59.045458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:55:59.045458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:55:59.045458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:55:59.045458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:55:59.045458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:55:59.045458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:55:59.045458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:55:59.045458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:55:59.046458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:55:59.046458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:55:59.046458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:55:59.046458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:55:59.046458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:55:59.046458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:55:59.046458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:55:59.046458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:55:59.046458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:55:59.046458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:55:59.046458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 21:55:59.046458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:55:59.046458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 612/1235] 10.222.0.44 () {56 vars in 1102 bytes} [Tue Jul 12 21:55:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:55:59.051893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40cde9-163a-4dea-915c-6a79f514a3f7 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:59.105467 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:59.168397 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40cde9-163a-4dea-915c-6a79f514a3f7 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:59.210022 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee2250-8ae2-4310-82ae-d2f57963cb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:59.211277 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee2250-8ae2-4310-82ae-d2f57963cb01 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:59.211567 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee2250-8ae2-4310-82ae-d2f57963cb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:59.211847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee2250-8ae2-4310-82ae-d2f57963cb01 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:59.212671 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee2250-8ae2-4310-82ae-d2f57963cb01 None admin] RBAC: Adding query filter params (name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:55:59.213023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee2250-8ae2-4310-82ae-d2f57963cb01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:59.214781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee2250-8ae2-4310-82ae-d2f57963cb01 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:59.225859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 624/1236] 10.222.0.44 () {56 vars in 1103 bytes} [Tue Jul 12 21:55:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:55:59.232474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214b56f-5fbb-4b25-a024-72ab0912f401 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:59.340917 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:59.360545 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214b56f-5fbb-4b25-a024-72ab0912f401 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:59.402761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724be909-f468-4f53-883b-3e752ca2216b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:59.403578 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724be909-f468-4f53-883b-3e752ca2216b None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:59.403873 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724be909-f468-4f53-883b-3e752ca2216b None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:59.404147 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724be909-f468-4f53-883b-3e752ca2216b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:59.416406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724be909-f468-4f53-883b-3e752ca2216b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:59.418192 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724be909-f468-4f53-883b-3e752ca2216b None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:59.441719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-724be909-f468-4f53-883b-3e752ca2216b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:55:59.441719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:55:59.441719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:55:59.441719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:55:59.441719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:55:59.441719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:55:59.441719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:55:59.441719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:55:59.441719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:55:59.441719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:55:59.441719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:55:59.441719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:55:59.441719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:55:59.441719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:55:59.441719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:55:59.442897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:55:59.442897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:55:59.442897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:55:59.442897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:55:59.442897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:55:59.442897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:55:59.442897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:55:59.442897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:55:59.442897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:55:59.442897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:55:59.442897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:55:59.442897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:55:59.442897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:55:59.442897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:55:59.442897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:55:59.442897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:55:59.442897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:55:59.443801 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:55:59.443801 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:55:59.443801 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:55:59.443801 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:55:59.443801 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 613/1237] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:55:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:55:59.449227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee2250-8ae2-4310-82ae-d2f57963cb01 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:59.562564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee2250-8ae2-4310-82ae-d2f57963cb01 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:59.572999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:59.602023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcaad56-6e3c-4bfc-a845-11a09947dcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:59.602713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcaad56-6e3c-4bfc-a845-11a09947dcfd None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:59.602997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcaad56-6e3c-4bfc-a845-11a09947dcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:59.603272 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcaad56-6e3c-4bfc-a845-11a09947dcfd None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:59.603987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcaad56-6e3c-4bfc-a845-11a09947dcfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:55:59.604292 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcaad56-6e3c-4bfc-a845-11a09947dcfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:59.605709 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcaad56-6e3c-4bfc-a845-11a09947dcfd None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:59.618024 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 625/1238] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:55:59.624185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724be909-f468-4f53-883b-3e752ca2216b None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:59.750821 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724be909-f468-4f53-883b-3e752ca2216b None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:55:59.796378 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad57ee6-a87a-42c7-afaa-6c29580eaf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:55:59.797260 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad57ee6-a87a-42c7-afaa-6c29580eaf47 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:55:59.797608 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad57ee6-a87a-42c7-afaa-6c29580eaf47 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:55:59.798031 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad57ee6-a87a-42c7-afaa-6c29580eaf47 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:55:59.801551 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:55:59.808138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad57ee6-a87a-42c7-afaa-6c29580eaf47 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:55:59.809928 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad57ee6-a87a-42c7-afaa-6c29580eaf47 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:55:59.830451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-4ad57ee6-a87a-42c7-afaa-6c29580eaf47 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 21:55:59.830451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:55:59.830451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:55:59.830451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:55:59.830451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:55:59.830451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:55:59.830451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:55:59.830451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:55:59.830451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:55:59.830451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:55:59.830451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:55:59.830451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:55:59.830451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:55:59.830451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:55:59.830451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:55:59.830451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:55:59.831694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:55:59.832645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:55:59.832645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:55:59.832645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:55:59.832645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:55:59.832645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:55:59.832645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:55:59.832645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:55:59.832645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:55:59.832645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 21:55:59.832645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:55:59.832645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 614/1239] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:55:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:55:59.838087 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcaad56-6e3c-4bfc-a845-11a09947dcfd None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:55:59.968894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcaad56-6e3c-4bfc-a845-11a09947dcfd None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:00.007370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944440b0-128f-4fb7-8490-3458488a8ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:00.008041 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944440b0-128f-4fb7-8490-3458488a8ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:00.008349 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944440b0-128f-4fb7-8490-3458488a8ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:00.008600 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944440b0-128f-4fb7-8490-3458488a8ce3 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:00.009561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944440b0-128f-4fb7-8490-3458488a8ce3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:00.009867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944440b0-128f-4fb7-8490-3458488a8ce3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:00.011487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944440b0-128f-4fb7-8490-3458488a8ce3 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:00.032892 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 626/1240] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Jul 12 21:55:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:56:00.039406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad57ee6-a87a-42c7-afaa-6c29580eaf47 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:00.154053 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad57ee6-a87a-42c7-afaa-6c29580eaf47 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:00.192463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54807e-a5c5-4565-8f50-72823b76ddf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:00.193410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54807e-a5c5-4565-8f50-72823b76ddf7 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:00.193695 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54807e-a5c5-4565-8f50-72823b76ddf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:00.193970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54807e-a5c5-4565-8f50-72823b76ddf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:00.259947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:00.268313 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54807e-a5c5-4565-8f50-72823b76ddf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:00.269873 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54807e-a5c5-4565-8f50-72823b76ddf7 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:00.389216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 615/1241] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:56:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:56:00.395249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944440b0-128f-4fb7-8490-3458488a8ce3 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:00.430097 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:00.513792 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944440b0-128f-4fb7-8490-3458488a8ce3 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:00.550891 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370dba22-a42b-4d81-a680-37fddea28157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:00.551757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dba22-a42b-4d81-a680-37fddea28157 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:00.552037 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dba22-a42b-4d81-a680-37fddea28157 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:00.552310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370dba22-a42b-4d81-a680-37fddea28157 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:00.564542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370dba22-a42b-4d81-a680-37fddea28157 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:00.566477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370dba22-a42b-4d81-a680-37fddea28157 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:00.585326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-370dba22-a42b-4d81-a680-37fddea28157 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:56:00.585326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:00.585326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:00.585326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:00.585326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:00.585326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:00.585326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:00.585326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:00.585326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:00.585326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:00.585326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:00.585326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:00.585326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:56:00.585326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:56:00.585326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:56:00.586516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:56:00.586516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:00.586516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:00.586516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:00.586516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:00.586516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:00.586516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:00.586516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:00.586516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:00.586516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:00.586516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:00.586516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:00.586516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:00.586516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:00.586516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:00.586516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:00.586516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:00.587598 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:56:00.587598 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:56:00.587598 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:56:00.587598 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:56:00.591178 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 627/1242] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:56:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:00.592867 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54807e-a5c5-4565-8f50-72823b76ddf7 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:00.702190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:00.709406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54807e-a5c5-4565-8f50-72823b76ddf7 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:00.752790 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fb929-8cc6-4dd7-a558-cfeee34cc1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:00.753499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fb929-8cc6-4dd7-a558-cfeee34cc1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:00.753725 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fb929-8cc6-4dd7-a558-cfeee34cc1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:00.754004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fb929-8cc6-4dd7-a558-cfeee34cc1a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:00.754727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960fb929-8cc6-4dd7-a558-cfeee34cc1a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:00.755035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960fb929-8cc6-4dd7-a558-cfeee34cc1a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:00.756415 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960fb929-8cc6-4dd7-a558-cfeee34cc1a6 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:00.768494 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 616/1243] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:00.774153 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370dba22-a42b-4d81-a680-37fddea28157 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:00.861401 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:00.885533 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370dba22-a42b-4d81-a680-37fddea28157 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:00.925262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea0769-b9ed-41dd-ab5b-0cdb97beb961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:00.926100 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea0769-b9ed-41dd-ab5b-0cdb97beb961 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:00.926385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea0769-b9ed-41dd-ab5b-0cdb97beb961 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:00.926662 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea0769-b9ed-41dd-ab5b-0cdb97beb961 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:00.933502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea0769-b9ed-41dd-ab5b-0cdb97beb961 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:00.934944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea0769-b9ed-41dd-ab5b-0cdb97beb961 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:00.949216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-3aea0769-b9ed-41dd-ab5b-0cdb97beb961 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:56:00.949216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:00.949216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:00.949216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:00.949216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:00.949216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:00.949216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:00.949216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:00.949216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:00.949216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:00.949216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:00.949216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:00.949216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:56:00.949216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:56:00.949216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:56:00.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:56:00.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:00.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:00.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:00.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:00.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:00.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:00.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:00.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:00.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:00.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:00.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:00.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:00.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:00.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:00.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:00.950583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:00.951713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:56:00.951713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:56:00.951713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:56:00.951713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:56:00.951713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:56:00.951713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:56:00.951713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:56:00.951713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:56:00.951713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 628/1244] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:56:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:00.956980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fb929-8cc6-4dd7-a558-cfeee34cc1a6 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:01.068046 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fb929-8cc6-4dd7-a558-cfeee34cc1a6 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:01.111424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63b24b-b456-4411-a898-b5e5e2876fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:01.112013 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63b24b-b456-4411-a898-b5e5e2876fae None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:01.112309 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63b24b-b456-4411-a898-b5e5e2876fae None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:01.112597 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63b24b-b456-4411-a898-b5e5e2876fae None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:01.113666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63b24b-b456-4411-a898-b5e5e2876fae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:01.113987 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63b24b-b456-4411-a898-b5e5e2876fae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:01.115831 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63b24b-b456-4411-a898-b5e5e2876fae None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:01.131397 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 617/1245] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:56:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:01.139112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea0769-b9ed-41dd-ab5b-0cdb97beb961 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:01.264440 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea0769-b9ed-41dd-ab5b-0cdb97beb961 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:01.302801 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b608236-4521-4e5b-b7e6-05e71f14d927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:01.303452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b608236-4521-4e5b-b7e6-05e71f14d927 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:01.303730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b608236-4521-4e5b-b7e6-05e71f14d927 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:01.303999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b608236-4521-4e5b-b7e6-05e71f14d927 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:01.304866 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b608236-4521-4e5b-b7e6-05e71f14d927 None admin] RBAC: Adding query filter params (role.id=f1e42ea5a1c440bba915b459d2744858, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=a1f1ef52f8eb42a59a50a69873b9ffaa) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:01.305196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b608236-4521-4e5b-b7e6-05e71f14d927 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:01.306606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b608236-4521-4e5b-b7e6-05e71f14d927 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:01.313404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 629/1246] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:56:01 2022] GET /identity/v3/role_assignments?user.id=a1f1ef52f8eb42a59a50a69873b9ffaa&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=f1e42ea5a1c440bba915b459d2744858 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:03.685447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bef09b-7428-4877-8abf-64646c35d3e4 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:03.686080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bef09b-7428-4877-8abf-64646c35d3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:03.686080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bef09b-7428-4877-8abf-64646c35d3e4 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:03.688100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 618/1247] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:56:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:56:03.694925 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7f63-a7b4-4d74-8ad5-aeeadf6257ba None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:03.695387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7f63-a7b4-4d74-8ad5-aeeadf6257ba None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:03.695499 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7f63-a7b4-4d74-8ad5-aeeadf6257ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:03.701305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:03.715276 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-407f7f63-a7b4-4d74-8ad5-aeeadf6257ba None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:56:03.716391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-407f7f63-a7b4-4d74-8ad5-aeeadf6257ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:56:03.800765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 630/1248] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:56:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:56:03.829855 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bef09b-7428-4877-8abf-64646c35d3e4 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:03.851462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:03.955329 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bef09b-7428-4877-8abf-64646c35d3e4 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:03.998919 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc777c26-0fd0-47b9-a601-d480dd7c3fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:03.999984 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc777c26-0fd0-47b9-a601-d480dd7c3fcb None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:04.000909 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc777c26-0fd0-47b9-a601-d480dd7c3fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:04.001261 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc777c26-0fd0-47b9-a601-d480dd7c3fcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:04.015174 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc777c26-0fd0-47b9-a601-d480dd7c3fcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:56:04.025513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-cc777c26-0fd0-47b9-a601-d480dd7c3fcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:56:04.025513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:04.025513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:04.025513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:04.025513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:04.025513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:04.025513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:04.025513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:04.025513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:04.025513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:04.025513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:04.025513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:04.025513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:56:04.025513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:56:04.025513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:56:04.025513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:56:04.026973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:56:04.026973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:56:04.026973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:56:04.026973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:56:04.026973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:04.026973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:04.026973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:04.026973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:04.026973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:04.026973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:04.026973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:04.026973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:04.026973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:04.026973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:04.026973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:04.026973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:04.028045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:04.028045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:04.028045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:04.028045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:04.028045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:56:04.028045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:56:04.028045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:56:04.028045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:56:04.028045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:56:04.028045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:56:04.028045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:56:04.028045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:56:04.028045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 619/1249] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:56:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:04.033845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f7f63-a7b4-4d74-8ad5-aeeadf6257ba None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:04.152305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f7f63-a7b4-4d74-8ad5-aeeadf6257ba None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:04.189584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:04.194921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177d899-706e-4143-b790-81e57182c2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:04.195812 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177d899-706e-4143-b790-81e57182c2fa None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:04.196140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177d899-706e-4143-b790-81e57182c2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:04.196606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177d899-706e-4143-b790-81e57182c2fa None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:04.197577 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177d899-706e-4143-b790-81e57182c2fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:04.198037 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177d899-706e-4143-b790-81e57182c2fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:04.199434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177d899-706e-4143-b790-81e57182c2fa None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:04.210138 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 631/1250] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:56:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:04.215975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc777c26-0fd0-47b9-a601-d480dd7c3fcb None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:04.303929 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:04.330347 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc777c26-0fd0-47b9-a601-d480dd7c3fcb None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:04.369853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265587c8-fa68-421e-8f2f-9c0f6f99c86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:04.370650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265587c8-fa68-421e-8f2f-9c0f6f99c86f None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:04.370935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265587c8-fa68-421e-8f2f-9c0f6f99c86f None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:04.371296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265587c8-fa68-421e-8f2f-9c0f6f99c86f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:04.385183 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265587c8-fa68-421e-8f2f-9c0f6f99c86f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:04.386851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265587c8-fa68-421e-8f2f-9c0f6f99c86f None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:04.406943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-265587c8-fa68-421e-8f2f-9c0f6f99c86f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:56:04.406943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:04.406943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:04.406943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:04.406943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:04.406943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:04.406943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:04.406943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:04.406943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:04.406943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:04.406943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:04.406943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:04.406943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:56:04.406943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:56:04.406943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:56:04.408083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:56:04.408083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:04.408083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:04.408083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:04.408083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:04.408083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:04.408083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:04.408083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:04.408083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:04.408083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:04.408083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:04.408083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:04.408083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:04.408083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:04.408083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:04.408083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:04.408083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:04.409172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:56:04.409172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:56:04.409172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:56:04.409172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:56:04.409172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 620/1251] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:56:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:04.414067 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177d899-706e-4143-b790-81e57182c2fa None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:04.532317 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177d899-706e-4143-b790-81e57182c2fa None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:04.569666 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638efa4d-035f-41a7-913c-f5b1089e6dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:04.570421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638efa4d-035f-41a7-913c-f5b1089e6dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:04.570773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638efa4d-035f-41a7-913c-f5b1089e6dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:04.571077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638efa4d-035f-41a7-913c-f5b1089e6dc1 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:04.571808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638efa4d-035f-41a7-913c-f5b1089e6dc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:04.572119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638efa4d-035f-41a7-913c-f5b1089e6dc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:04.573531 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638efa4d-035f-41a7-913c-f5b1089e6dc1 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:04.590513 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 632/1252] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:04.596517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265587c8-fa68-421e-8f2f-9c0f6f99c86f None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:04.721734 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265587c8-fa68-421e-8f2f-9c0f6f99c86f None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:04.755616 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:04.766509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c0127-b74b-43be-9b2b-ca6bc2cf8cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:04.767733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c0127-b74b-43be-9b2b-ca6bc2cf8cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:04.768183 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c0127-b74b-43be-9b2b-ca6bc2cf8cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:04.768640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c0127-b74b-43be-9b2b-ca6bc2cf8cd5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:04.777302 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c0127-b74b-43be-9b2b-ca6bc2cf8cd5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:04.778767 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c0127-b74b-43be-9b2b-ca6bc2cf8cd5 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:04.796307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-520c0127-b74b-43be-9b2b-ca6bc2cf8cd5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 21:56:04.796307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:04.796307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:04.796307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:04.796307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:04.796307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:04.796307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:04.796307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:04.796307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:04.796307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:04.796307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:04.796307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:04.796307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:56:04.796307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:56:04.796307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:56:04.796307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:04.798492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:04.799623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:04.799623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:04.799623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:56:04.799623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:56:04.799623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:56:04.799623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:56:04.799623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:56:04.799623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:56:04.799623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 21:56:04.799623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:56:04.799623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 621/1253] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:56:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:04.804007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638efa4d-035f-41a7-913c-f5b1089e6dc1 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:04.864286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:04.913489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638efa4d-035f-41a7-913c-f5b1089e6dc1 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:04.949413 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73936b9e-25fd-44a8-b7c5-840c7a795e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:04.950158 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73936b9e-25fd-44a8-b7c5-840c7a795e72 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:04.950481 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73936b9e-25fd-44a8-b7c5-840c7a795e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:04.950763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73936b9e-25fd-44a8-b7c5-840c7a795e72 None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:04.951608 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73936b9e-25fd-44a8-b7c5-840c7a795e72 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:04.951923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73936b9e-25fd-44a8-b7c5-840c7a795e72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:04.953451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73936b9e-25fd-44a8-b7c5-840c7a795e72 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:04.974303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 633/1254] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Jul 12 21:56:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:56:04.980527 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c0127-b74b-43be-9b2b-ca6bc2cf8cd5 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:05.107265 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c0127-b74b-43be-9b2b-ca6bc2cf8cd5 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:05.146045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780f4bc-04b8-4f54-86d4-ebdd35c7f24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:05.146694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780f4bc-04b8-4f54-86d4-ebdd35c7f24c None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:05.146977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780f4bc-04b8-4f54-86d4-ebdd35c7f24c None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:05.147251 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780f4bc-04b8-4f54-86d4-ebdd35c7f24c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:05.212574 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:05.226062 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780f4bc-04b8-4f54-86d4-ebdd35c7f24c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:05.227527 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780f4bc-04b8-4f54-86d4-ebdd35c7f24c None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:05.336065 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 622/1255] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:56:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 21:56:05.337487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73936b9e-25fd-44a8-b7c5-840c7a795e72 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:05.447006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73936b9e-25fd-44a8-b7c5-840c7a795e72 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:05.461339 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:05.485346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f9d87-fa47-4b64-8329-cd9ba4077a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:05.486145 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f9d87-fa47-4b64-8329-cd9ba4077a3a None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:05.486422 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f9d87-fa47-4b64-8329-cd9ba4077a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:05.486688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f9d87-fa47-4b64-8329-cd9ba4077a3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:05.497086 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f9d87-fa47-4b64-8329-cd9ba4077a3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:05.498546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f9d87-fa47-4b64-8329-cd9ba4077a3a None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:05.518334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-893f9d87-fa47-4b64-8329-cd9ba4077a3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:56:05.518334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:05.518334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:05.518334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:05.518334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:05.518334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:05.518334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:05.518334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:05.518334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:05.518334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:05.518334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:05.518334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:05.518334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:56:05.518334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:56:05.518334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:56:05.519507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:56:05.519507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:05.519507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:05.519507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:05.519507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:05.519507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:05.519507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:05.519507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:05.519507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:05.519507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:05.519507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:05.519507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:05.519507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:05.519507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:05.519507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:05.519507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:05.519507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:05.520752 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:56:05.520752 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:56:05.520752 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:56:05.520752 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:56:05.528527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 634/1256] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:05.528688 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780f4bc-04b8-4f54-86d4-ebdd35c7f24c None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:05.621665 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:05.635610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780f4bc-04b8-4f54-86d4-ebdd35c7f24c None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:05.680612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9eb483-5055-411f-b57f-91fc098db775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:05.681580 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9eb483-5055-411f-b57f-91fc098db775 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:05.681985 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9eb483-5055-411f-b57f-91fc098db775 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:05.682330 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9eb483-5055-411f-b57f-91fc098db775 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:05.683574 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9eb483-5055-411f-b57f-91fc098db775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:05.684103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9eb483-5055-411f-b57f-91fc098db775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:05.686365 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9eb483-5055-411f-b57f-91fc098db775 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:05.703266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 623/1257] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:05.710191 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f9d87-fa47-4b64-8329-cd9ba4077a3a None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:05.830165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f9d87-fa47-4b64-8329-cd9ba4077a3a None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:05.866304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746f4cd-2dd1-4c9e-a906-9249bdfe7ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:05.867095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746f4cd-2dd1-4c9e-a906-9249bdfe7ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:05.867375 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746f4cd-2dd1-4c9e-a906-9249bdfe7ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:05.867718 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746f4cd-2dd1-4c9e-a906-9249bdfe7ea8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:05.874587 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746f4cd-2dd1-4c9e-a906-9249bdfe7ea8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:05.876410 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746f4cd-2dd1-4c9e-a906-9249bdfe7ea8 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:05.893108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-0746f4cd-2dd1-4c9e-a906-9249bdfe7ea8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:56:05.893108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:05.893108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:05.893108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:05.893108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:05.893108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:05.893108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:05.893108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:05.893108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:05.893108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:05.893108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:05.893108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:05.893108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:56:05.893108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:56:05.893108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:56:05.894361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:56:05.894361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:05.894361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:05.894361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:05.894361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:05.894361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:05.894361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:05.894361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:05.894361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:05.894361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:05.894361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:05.894361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:05.894361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:05.894361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:05.894361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:05.894361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:05.894361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:05.900321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:56:05.900321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:56:05.900321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:56:05.900321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:56:05.900321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:56:05.900321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:56:05.900321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:56:05.900321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:56:05.900321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 635/1258] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:56:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:05.900955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9eb483-5055-411f-b57f-91fc098db775 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:06.007392 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9eb483-5055-411f-b57f-91fc098db775 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:06.045970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d265fd-15a2-4416-8742-2f58ddceb013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:06.046654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d265fd-15a2-4416-8742-2f58ddceb013 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:06.046931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d265fd-15a2-4416-8742-2f58ddceb013 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:06.047239 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d265fd-15a2-4416-8742-2f58ddceb013 None admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:06.048196 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d265fd-15a2-4416-8742-2f58ddceb013 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:06.048526 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d265fd-15a2-4416-8742-2f58ddceb013 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:06.050138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d265fd-15a2-4416-8742-2f58ddceb013 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:06.064442 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 624/1259] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:56:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:06.071160 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746f4cd-2dd1-4c9e-a906-9249bdfe7ea8 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:06.101385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:06.184792 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746f4cd-2dd1-4c9e-a906-9249bdfe7ea8 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:06.223803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d0e36-1ead-49ec-b3cc-d6a654a0df44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:06.224392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d0e36-1ead-49ec-b3cc-d6a654a0df44 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:06.224670 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d0e36-1ead-49ec-b3cc-d6a654a0df44 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:06.224977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d0e36-1ead-49ec-b3cc-d6a654a0df44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:06.225827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d0e36-1ead-49ec-b3cc-d6a654a0df44 None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=a1f1ef52f8eb42a59a50a69873b9ffaa) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:06.226141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d0e36-1ead-49ec-b3cc-d6a654a0df44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:06.227620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d0e36-1ead-49ec-b3cc-d6a654a0df44 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:06.244746 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 636/1260] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:56:06 2022] GET /identity/v3/role_assignments?user.id=a1f1ef52f8eb42a59a50a69873b9ffaa&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=053d1730ed764973abb1490894e07681 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:08.276909 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f4ea2-699d-4d37-84f9-f972a48ac593 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:08.277429 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f4ea2-699d-4d37-84f9-f972a48ac593 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:08.277523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f4ea2-699d-4d37-84f9-f972a48ac593 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:08.278844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 625/1261] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:56:08.284524 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c2e2f-a0cc-4617-9e58-c0b30bf9486e None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:08.284975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c2e2f-a0cc-4617-9e58-c0b30bf9486e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:08.285111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c2e2f-a0cc-4617-9e58-c0b30bf9486e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:08.301774 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-381c2e2f-a0cc-4617-9e58-c0b30bf9486e None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:56:08.302979 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-381c2e2f-a0cc-4617-9e58-c0b30bf9486e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:56:08.368627 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 637/1262] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:56:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:56:08.391963 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f4ea2-699d-4d37-84f9-f972a48ac593 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:08.402718 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:08.497915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f4ea2-699d-4d37-84f9-f972a48ac593 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:08.530934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838e98a-b15d-457c-8f8c-c821dc00bf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:08.532215 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838e98a-b15d-457c-8f8c-c821dc00bf72 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:08.532499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838e98a-b15d-457c-8f8c-c821dc00bf72 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:08.532801 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838e98a-b15d-457c-8f8c-c821dc00bf72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:08.542627 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838e98a-b15d-457c-8f8c-c821dc00bf72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:56:08.550395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-7838e98a-b15d-457c-8f8c-c821dc00bf72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:56:08.550395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:08.550395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:08.550395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:08.550395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:08.550395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:08.550395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:08.550395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:08.550395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:08.550395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:08.550395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:08.550395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:08.550395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:56:08.550395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:56:08.550395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:56:08.550395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:56:08.551492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:56:08.551492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:56:08.551492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:56:08.551492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:56:08.551492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:08.551492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:08.551492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:08.551492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:08.551492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:08.551492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:08.551492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:08.551492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:08.551492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:08.551492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:08.551492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:08.551492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:08.552630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:08.552630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:08.552630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:08.552630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:08.552630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:56:08.552630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:56:08.552630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:56:08.552630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:56:08.552630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:56:08.552630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:56:08.552630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:56:08.552630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:56:08.552630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 626/1263] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:56:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:08.557734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c2e2f-a0cc-4617-9e58-c0b30bf9486e None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:08.575846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:08.656480 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c2e2f-a0cc-4617-9e58-c0b30bf9486e None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:08.696357 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a53686-ef5f-42c2-b914-22db40920031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:08.697115 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a53686-ef5f-42c2-b914-22db40920031 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:08.697466 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a53686-ef5f-42c2-b914-22db40920031 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:08.697811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a53686-ef5f-42c2-b914-22db40920031 None admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:08.698725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a53686-ef5f-42c2-b914-22db40920031 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:08.699076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a53686-ef5f-42c2-b914-22db40920031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:08.700687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a53686-ef5f-42c2-b914-22db40920031 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:08.710783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 638/1264] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:56:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:08.717250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838e98a-b15d-457c-8f8c-c821dc00bf72 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:08.793402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:08.827370 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838e98a-b15d-457c-8f8c-c821dc00bf72 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:08.860180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12ba2f-f8cb-46de-a755-8a117f6414a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:08.860994 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12ba2f-f8cb-46de-a755-8a117f6414a3 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:08.861278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12ba2f-f8cb-46de-a755-8a117f6414a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:08.861553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12ba2f-f8cb-46de-a755-8a117f6414a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:08.871868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12ba2f-f8cb-46de-a755-8a117f6414a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:08.873338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12ba2f-f8cb-46de-a755-8a117f6414a3 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:08.891742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-3a12ba2f-f8cb-46de-a755-8a117f6414a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:56:08.891742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:08.891742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:08.891742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:08.891742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:08.891742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:08.891742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:08.891742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:08.891742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:08.891742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:08.891742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:08.891742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:08.891742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:56:08.891742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:56:08.891742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:56:08.892913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:56:08.892913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:08.892913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:08.892913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:08.892913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:08.892913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:08.892913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:08.892913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:08.892913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:08.892913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:08.892913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:08.892913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:08.892913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:08.892913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:08.892913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:08.892913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:08.892913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:08.894151 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:56:08.894151 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:56:08.894151 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:56:08.894151 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:56:08.894151 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 627/1265] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:56:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:08.898902 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a53686-ef5f-42c2-b914-22db40920031 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:08.979492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:08.998383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a53686-ef5f-42c2-b914-22db40920031 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:09.030538 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1f2d6-9584-4f64-82b6-76953b517219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:09.031210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1f2d6-9584-4f64-82b6-76953b517219 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:09.031489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1f2d6-9584-4f64-82b6-76953b517219 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:09.031761 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1f2d6-9584-4f64-82b6-76953b517219 None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:09.032468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1f2d6-9584-4f64-82b6-76953b517219 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:09.032798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1f2d6-9584-4f64-82b6-76953b517219 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:09.034162 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1f2d6-9584-4f64-82b6-76953b517219 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:09.045466 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 639/1266] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 21:56:09.051710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12ba2f-f8cb-46de-a755-8a117f6414a3 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:09.160456 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12ba2f-f8cb-46de-a755-8a117f6414a3 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:09.179817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:09.194676 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5a7a2-1d9d-4853-8d0a-78c11491a156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:09.195469 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5a7a2-1d9d-4853-8d0a-78c11491a156 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:09.195750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5a7a2-1d9d-4853-8d0a-78c11491a156 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:09.196022 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5a7a2-1d9d-4853-8d0a-78c11491a156 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:09.203442 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5a7a2-1d9d-4853-8d0a-78c11491a156 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:09.204936 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5a7a2-1d9d-4853-8d0a-78c11491a156 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:09.218793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-76b5a7a2-1d9d-4853-8d0a-78c11491a156 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 21:56:09.218793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:09.218793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:09.218793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:09.218793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:09.218793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:09.218793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:09.218793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:09.218793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:09.218793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:09.218793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:09.218793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:09.218793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:56:09.218793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:56:09.218793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:56:09.218793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:09.219728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:09.220925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:09.220925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:09.220925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:56:09.220925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:56:09.220925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:56:09.220925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:56:09.220925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:56:09.220925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:56:09.220925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 21:56:09.220925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:56:09.220925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 628/1267] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:56:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:09.226010 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1f2d6-9584-4f64-82b6-76953b517219 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:09.323331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1f2d6-9584-4f64-82b6-76953b517219 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:09.356766 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c3d84-0b7b-44a6-af06-96a3b249b80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:09.357429 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c3d84-0b7b-44a6-af06-96a3b249b80f None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:09.357710 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c3d84-0b7b-44a6-af06-96a3b249b80f None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:09.357986 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c3d84-0b7b-44a6-af06-96a3b249b80f None admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:09.358813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c3d84-0b7b-44a6-af06-96a3b249b80f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:09.359122 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c3d84-0b7b-44a6-af06-96a3b249b80f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:09.360693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c3d84-0b7b-44a6-af06-96a3b249b80f None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:09.369048 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:09.377988 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 640/1268] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Jul 12 21:56:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:09.383800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5a7a2-1d9d-4853-8d0a-78c11491a156 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:09.485603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5a7a2-1d9d-4853-8d0a-78c11491a156 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:09.518126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa78cc-ba6d-4def-96fb-c4a3c3d19f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:09.518897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa78cc-ba6d-4def-96fb-c4a3c3d19f2e None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:09.519176 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa78cc-ba6d-4def-96fb-c4a3c3d19f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:09.519444 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa78cc-ba6d-4def-96fb-c4a3c3d19f2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:09.529702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa78cc-ba6d-4def-96fb-c4a3c3d19f2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:09.531666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa78cc-ba6d-4def-96fb-c4a3c3d19f2e None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:09.548593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-b3fa78cc-ba6d-4def-96fb-c4a3c3d19f2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:56:09.548593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:09.548593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:09.548593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:09.548593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:09.548593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:09.548593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:09.548593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:09.548593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:09.548593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:09.548593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:09.548593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:09.548593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:56:09.548593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:56:09.548593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:56:09.549675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:56:09.549675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:09.549675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:09.549675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:09.549675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:09.549675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:09.549675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:09.549675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:09.549675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:09.549675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:09.549675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:09.549675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:09.549675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:09.549675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:09.549675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:09.549675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:09.549675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:09.550809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:56:09.550809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:56:09.550809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:56:09.550809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:56:09.550809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 629/1269] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:09.555599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c3d84-0b7b-44a6-af06-96a3b249b80f None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:09.654929 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c3d84-0b7b-44a6-af06-96a3b249b80f None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:09.695704 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3132b-a4a2-424a-bbef-9db91472fe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:09.696363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3132b-a4a2-424a-bbef-9db91472fe5e None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:09.696650 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3132b-a4a2-424a-bbef-9db91472fe5e None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:09.696960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3132b-a4a2-424a-bbef-9db91472fe5e None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:09.697685 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3132b-a4a2-424a-bbef-9db91472fe5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:09.697994 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3132b-a4a2-424a-bbef-9db91472fe5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:09.699379 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3132b-a4a2-424a-bbef-9db91472fe5e None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:09.710623 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 641/1270] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:09.716262 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa78cc-ba6d-4def-96fb-c4a3c3d19f2e None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:09.748146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:09.815189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa78cc-ba6d-4def-96fb-c4a3c3d19f2e None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:09.850103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cc132-b778-4296-ab19-afa8851aa632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:09.851398 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cc132-b778-4296-ab19-afa8851aa632 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:09.851775 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cc132-b778-4296-ab19-afa8851aa632 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:09.852060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cc132-b778-4296-ab19-afa8851aa632 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:09.859556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51cc132-b778-4296-ab19-afa8851aa632 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:09.861029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51cc132-b778-4296-ab19-afa8851aa632 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:09.875126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-a51cc132-b778-4296-ab19-afa8851aa632 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:56:09.875126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:09.875126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:09.875126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:09.875126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:09.875126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:09.875126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:09.875126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:09.875126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:09.875126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:09.875126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:09.875126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:09.875126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:56:09.875126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:56:09.875126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:56:09.876162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:56:09.876162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:09.876162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:09.876162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:09.876162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:09.876162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:09.876162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:09.876162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:09.876162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:09.876162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:09.876162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:09.876162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:09.876162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:09.876162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:09.876162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:09.876162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:09.876162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:09.877233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:56:09.877233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:56:09.877233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:56:09.877233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:56:09.877233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:56:09.877233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:56:09.877233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:56:09.877233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:56:09.877233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 630/1271] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:56:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:09.882225 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3132b-a4a2-424a-bbef-9db91472fe5e None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:09.940453 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:09.983055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3132b-a4a2-424a-bbef-9db91472fe5e None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:10.016489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115f918-f4ff-4357-909c-df888e187f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:10.017169 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115f918-f4ff-4357-909c-df888e187f28 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:10.017458 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115f918-f4ff-4357-909c-df888e187f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:10.017739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115f918-f4ff-4357-909c-df888e187f28 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:10.018525 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115f918-f4ff-4357-909c-df888e187f28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:10.018841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115f918-f4ff-4357-909c-df888e187f28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:10.020276 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115f918-f4ff-4357-909c-df888e187f28 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:10.032155 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 642/1272] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:56:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:10.038063 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cc132-b778-4296-ab19-afa8851aa632 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:10.129509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:10.138648 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cc132-b778-4296-ab19-afa8851aa632 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:10.171370 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f718be0-0da6-4978-86bc-fd63d109e952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:10.172850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f718be0-0da6-4978-86bc-fd63d109e952 None admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:10.173128 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f718be0-0da6-4978-86bc-fd63d109e952 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:10.173401 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f718be0-0da6-4978-86bc-fd63d109e952 None admin] PATH_INFO: `/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/a1f1ef52f8eb42a59a50a69873b9ffaa/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:10.175329 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f718be0-0da6-4978-86bc-fd63d109e952 None admin] RBAC: Authorizing `identity:create_grant(project_id=459e4743ff3043f69408ca2095be3bd0, user_id=a1f1ef52f8eb42a59a50a69873b9ffaa, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:10.176775 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f718be0-0da6-4978-86bc-fd63d109e952 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:10.189760 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 631/1273] 10.222.0.44 () {58 vars in 1439 bytes} [Tue Jul 12 21:56:10 2022] PUT /identity/v3/projects/459e4743ff3043f69408ca2095be3bd0/users/a1f1ef52f8eb42a59a50a69873b9ffaa/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 21:56:12.233777 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab216bcf-00f6-4ef3-86aa-f712d26eedf0 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:12.234358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab216bcf-00f6-4ef3-86aa-f712d26eedf0 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:12.234467 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab216bcf-00f6-4ef3-86aa-f712d26eedf0 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:12.235862 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 643/1274] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:56:12.240981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3ad21-055a-44f0-bf95-991b28a41763 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:12.241409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3ad21-055a-44f0-bf95-991b28a41763 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:12.241553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3ad21-055a-44f0-bf95-991b28a41763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:12.261772 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-b6e3ad21-055a-44f0-bf95-991b28a41763 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:56:12.262972 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b6e3ad21-055a-44f0-bf95-991b28a41763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:56:12.332783 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 632/1275] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:56:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:56:12.356892 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab216bcf-00f6-4ef3-86aa-f712d26eedf0 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:12.470676 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab216bcf-00f6-4ef3-86aa-f712d26eedf0 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:12.488438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:12.505310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288dcd9b-d505-4bdd-bf39-08cfc4c0e6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:12.506097 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288dcd9b-d505-4bdd-bf39-08cfc4c0e6de None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:12.506379 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288dcd9b-d505-4bdd-bf39-08cfc4c0e6de None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:12.506654 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288dcd9b-d505-4bdd-bf39-08cfc4c0e6de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:12.517128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288dcd9b-d505-4bdd-bf39-08cfc4c0e6de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 21:56:12.524667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-288dcd9b-d505-4bdd-bf39-08cfc4c0e6de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:56:12.524667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:12.524667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:12.524667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:12.524667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:12.524667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:12.524667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:12.524667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:12.524667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:12.524667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:12.524667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:12.524667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:12.524667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 21:56:12.524667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 21:56:12.524667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 21:56:12.524667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 21:56:12.525854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 21:56:12.525854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 21:56:12.525854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 21:56:12.525854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 21:56:12.525854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:12.525854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:12.525854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:12.525854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:12.525854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:12.525854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:12.525854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:12.525854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:12.525854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:12.525854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:12.525854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:12.525854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:12.526913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:12.526913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:12.526913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:12.526913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:12.526913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 21:56:12.526913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 21:56:12.526913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 21:56:12.526913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 21:56:12.526913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 21:56:12.526913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 21:56:12.526913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 21:56:12.526913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:56:12.526913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 644/1276] 10.222.0.44 () {56 vars in 1096 bytes} [Tue Jul 12 21:56:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:12.531926 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3ad21-055a-44f0-bf95-991b28a41763 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:12.610995 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:12.630245 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3ad21-055a-44f0-bf95-991b28a41763 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:12.666577 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b885ac-4261-4e55-8a47-1ddaddbfae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:12.667426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b885ac-4261-4e55-8a47-1ddaddbfae63 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:12.667777 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b885ac-4261-4e55-8a47-1ddaddbfae63 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:12.668068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b885ac-4261-4e55-8a47-1ddaddbfae63 None admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:12.669002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b885ac-4261-4e55-8a47-1ddaddbfae63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:12.669384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b885ac-4261-4e55-8a47-1ddaddbfae63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:12.671073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b885ac-4261-4e55-8a47-1ddaddbfae63 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:12.682202 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 633/1277] 10.222.0.44 () {56 vars in 1099 bytes} [Tue Jul 12 21:56:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:12.688350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288dcd9b-d505-4bdd-bf39-08cfc4c0e6de None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:12.792567 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288dcd9b-d505-4bdd-bf39-08cfc4c0e6de None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:12.824837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5e605-a3b3-4c43-a5a0-323dd2ecf400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:12.825616 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5e605-a3b3-4c43-a5a0-323dd2ecf400 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:12.825900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5e605-a3b3-4c43-a5a0-323dd2ecf400 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:12.826174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5e605-a3b3-4c43-a5a0-323dd2ecf400 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:12.837109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5e605-a3b3-4c43-a5a0-323dd2ecf400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:12.838562 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5e605-a3b3-4c43-a5a0-323dd2ecf400 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:12.855867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-5ef5e605-a3b3-4c43-a5a0-323dd2ecf400 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:56:12.855867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:12.855867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:12.855867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:12.855867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:12.855867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:12.855867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:12.855867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:12.855867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:12.855867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:12.855867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:12.855867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:12.855867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:56:12.855867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:56:12.855867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:56:12.857042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:56:12.857042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:12.857042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:12.857042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:12.857042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:12.857042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:12.857042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:12.857042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:12.857042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:12.857042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:12.857042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:12.857042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:12.857042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:12.857042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:12.857042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:12.857042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:12.857042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:12.858218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:56:12.858218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:56:12.858218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:56:12.858218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:56:12.858218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 645/1278] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:12.862944 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b885ac-4261-4e55-8a47-1ddaddbfae63 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:12.961749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b885ac-4261-4e55-8a47-1ddaddbfae63 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:12.995508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac687ea1-fff3-4cb9-b7bc-6c2da80b52d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:12.996159 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac687ea1-fff3-4cb9-b7bc-6c2da80b52d5 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:12.996444 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac687ea1-fff3-4cb9-b7bc-6c2da80b52d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:12.996750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac687ea1-fff3-4cb9-b7bc-6c2da80b52d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:12.997466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac687ea1-fff3-4cb9-b7bc-6c2da80b52d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:12.997772 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac687ea1-fff3-4cb9-b7bc-6c2da80b52d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:12.999134 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac687ea1-fff3-4cb9-b7bc-6c2da80b52d5 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:13.010621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 634/1279] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:13.016962 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5e605-a3b3-4c43-a5a0-323dd2ecf400 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:13.047208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:13.114729 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5e605-a3b3-4c43-a5a0-323dd2ecf400 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:13.147918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fbcde-a89a-4aee-b46a-77b111fe8a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:13.148735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fbcde-a89a-4aee-b46a-77b111fe8a01 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:13.149020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fbcde-a89a-4aee-b46a-77b111fe8a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:13.149295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fbcde-a89a-4aee-b46a-77b111fe8a01 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:13.157281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fbcde-a89a-4aee-b46a-77b111fe8a01 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:13.158726 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fbcde-a89a-4aee-b46a-77b111fe8a01 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:13.173451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-9a0fbcde-a89a-4aee-b46a-77b111fe8a01 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 21:56:13.173451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:13.173451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:13.173451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:13.173451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:13.173451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:13.173451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:13.173451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:13.173451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:13.173451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:13.173451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:13.173451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:13.173451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 21:56:13.173451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 21:56:13.173451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 21:56:13.173451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:13.174576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:13.175842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:13.175842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:13.175842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 21:56:13.175842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 21:56:13.175842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 21:56:13.175842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 21:56:13.175842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 21:56:13.175842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 21:56:13.175842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 21:56:13.175842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:56:13.175842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 646/1280] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:56:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:13.180507 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac687ea1-fff3-4cb9-b7bc-6c2da80b52d5 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:13.184306 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:13.277338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac687ea1-fff3-4cb9-b7bc-6c2da80b52d5 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:13.311136 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524ec3a-88c1-4f88-891c-cd2f40d043b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:13.311732 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524ec3a-88c1-4f88-891c-cd2f40d043b1 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:13.312020 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524ec3a-88c1-4f88-891c-cd2f40d043b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:13.312422 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524ec3a-88c1-4f88-891c-cd2f40d043b1 None admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:13.313178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524ec3a-88c1-4f88-891c-cd2f40d043b1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:13.313494 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524ec3a-88c1-4f88-891c-cd2f40d043b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:13.314923 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524ec3a-88c1-4f88-891c-cd2f40d043b1 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:13.332043 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 635/1281] 10.222.0.44 () {56 vars in 1143 bytes} [Tue Jul 12 21:56:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:13.337754 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fbcde-a89a-4aee-b46a-77b111fe8a01 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:13.426993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:13.436948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fbcde-a89a-4aee-b46a-77b111fe8a01 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:13.470382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a1b74-985f-4b5b-affd-aa6d12d03ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:13.471018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a1b74-985f-4b5b-affd-aa6d12d03ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:13.471298 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a1b74-985f-4b5b-affd-aa6d12d03ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:13.471570 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a1b74-985f-4b5b-affd-aa6d12d03ed1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:13.537408 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a1b74-985f-4b5b-affd-aa6d12d03ed1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:13.538848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a1b74-985f-4b5b-affd-aa6d12d03ed1 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:13.623352 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:13.633894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 647/1282] 10.222.0.44 () {58 vars in 1282 bytes} [Tue Jul 12 21:56:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 21:56:13.639655 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524ec3a-88c1-4f88-891c-cd2f40d043b1 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:13.706688 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:13.744992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524ec3a-88c1-4f88-891c-cd2f40d043b1 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:13.778297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47431e6-3344-42f8-ad9c-61d909e006a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:13.779090 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47431e6-3344-42f8-ad9c-61d909e006a2 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:13.779375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47431e6-3344-42f8-ad9c-61d909e006a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:13.779652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47431e6-3344-42f8-ad9c-61d909e006a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:13.790063 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47431e6-3344-42f8-ad9c-61d909e006a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:13.791500 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47431e6-3344-42f8-ad9c-61d909e006a2 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:13.808997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-e47431e6-3344-42f8-ad9c-61d909e006a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:56:13.808997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:13.808997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:13.808997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:13.808997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:13.808997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:13.808997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:13.808997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:13.808997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:13.808997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:13.808997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:13.808997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:13.808997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 21:56:13.808997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 21:56:13.808997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 21:56:13.810060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 21:56:13.810060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:13.810060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:13.810060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:13.810060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:13.810060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:13.810060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:13.810060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:13.810060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:13.810060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:13.810060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:13.810060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:13.810060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:13.810060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:13.810060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:13.810060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:13.810060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:13.811007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 21:56:13.811007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 21:56:13.811007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 21:56:13.811007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:56:13.811007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 636/1283] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 21:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:13.816375 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a1b74-985f-4b5b-affd-aa6d12d03ed1 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:13.917851 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a1b74-985f-4b5b-affd-aa6d12d03ed1 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:13.951572 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812d177-2f72-45af-bffb-412f72f6b7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:13.952236 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812d177-2f72-45af-bffb-412f72f6b7db None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:13.952521 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812d177-2f72-45af-bffb-412f72f6b7db None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:13.953421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812d177-2f72-45af-bffb-412f72f6b7db None admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:13.954147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812d177-2f72-45af-bffb-412f72f6b7db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:13.954457 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812d177-2f72-45af-bffb-412f72f6b7db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:13.955837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812d177-2f72-45af-bffb-412f72f6b7db None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:13.970302 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 648/1284] 10.222.0.44 () {56 vars in 1109 bytes} [Tue Jul 12 21:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:13.975964 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47431e6-3344-42f8-ad9c-61d909e006a2 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:14.073694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47431e6-3344-42f8-ad9c-61d909e006a2 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:14.088009 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:14.107957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73e310-659d-4bb9-b537-8f23b2fbfc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:14.108765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e310-659d-4bb9-b537-8f23b2fbfc57 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:14.109083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e310-659d-4bb9-b537-8f23b2fbfc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:14.109402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e310-659d-4bb9-b537-8f23b2fbfc57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:14.115914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73e310-659d-4bb9-b537-8f23b2fbfc57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:14.117391 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73e310-659d-4bb9-b537-8f23b2fbfc57 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:14.131247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application [None req-8c73e310-659d-4bb9-b537-8f23b2fbfc57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:56:14.131247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:14.131247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:14.131247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:14.131247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:14.131247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:14.131247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:14.131247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:14.131247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:14.131247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:14.131247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:14.131247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:14.131247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 21:56:14.131247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 21:56:14.131247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 21:56:14.132310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 21:56:14.132310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:14.132310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:14.132310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:14.132310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:14.132310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:14.132310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:14.132310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:14.132310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:14.132310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:14.132310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:14.132310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:14.132310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:14.132310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:14.132310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:14.132310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:14.132310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:14.133433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 21:56:14.133433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 21:56:14.133433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 21:56:14.133433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 21:56:14.133433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 21:56:14.133433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 21:56:14.133433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 21:56:14.133433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: ERROR keystone.server.flask.application Jul 12 21:56:14.133433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 637/1285] 10.222.0.44 () {56 vars in 1111 bytes} [Tue Jul 12 21:56:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:14.138254 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812d177-2f72-45af-bffb-412f72f6b7db None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:14.186545 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:14.236893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812d177-2f72-45af-bffb-412f72f6b7db None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:14.272798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cf096-cab2-4bf0-8269-788f891cccd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:14.273453 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cf096-cab2-4bf0-8269-788f891cccd2 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:14.273739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cf096-cab2-4bf0-8269-788f891cccd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:14.274016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cf096-cab2-4bf0-8269-788f891cccd2 None admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:14.274803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cf096-cab2-4bf0-8269-788f891cccd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:14.275117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cf096-cab2-4bf0-8269-788f891cccd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:14.276548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cf096-cab2-4bf0-8269-788f891cccd2 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:14.289520 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 649/1286] 10.222.0.44 () {56 vars in 1148 bytes} [Tue Jul 12 21:56:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:14.295502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73e310-659d-4bb9-b537-8f23b2fbfc57 None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:14.393850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73e310-659d-4bb9-b537-8f23b2fbfc57 None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:14.427326 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64078c33-0459-4501-9184-48c26665e69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:14.427977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64078c33-0459-4501-9184-48c26665e69b None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:14.428256 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64078c33-0459-4501-9184-48c26665e69b None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:14.428532 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64078c33-0459-4501-9184-48c26665e69b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:14.429386 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64078c33-0459-4501-9184-48c26665e69b None admin] RBAC: Adding query filter params (role.id=053d1730ed764973abb1490894e07681, scope.project.id=459e4743ff3043f69408ca2095be3bd0, user.id=a1f1ef52f8eb42a59a50a69873b9ffaa) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:14.429701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64078c33-0459-4501-9184-48c26665e69b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:14.431134 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64078c33-0459-4501-9184-48c26665e69b None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:14.436944 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 638/1287] 10.222.0.44 () {56 vars in 1374 bytes} [Tue Jul 12 21:56:14 2022] GET /identity/v3/role_assignments?user.id=a1f1ef52f8eb42a59a50a69873b9ffaa&scope.project.id=459e4743ff3043f69408ca2095be3bd0&role.id=053d1730ed764973abb1490894e07681 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:16.480382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a1316-a6bd-4c7f-8192-b8809b850eec None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:16.480946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a1316-a6bd-4c7f-8192-b8809b850eec None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:16.481061 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a1316-a6bd-4c7f-8192-b8809b850eec None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:16.482345 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 650/1288] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:56:16.487962 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e419a8f-673d-43ed-922b-1fb4627605f1 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:16.488406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e419a8f-673d-43ed-922b-1fb4627605f1 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:16.488526 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e419a8f-673d-43ed-922b-1fb4627605f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:16.505767 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-8e419a8f-673d-43ed-922b-1fb4627605f1 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:56:16.506968 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8e419a8f-673d-43ed-922b-1fb4627605f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:56:16.535483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:16.573652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 639/1289] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:56:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:56:16.595877 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a1316-a6bd-4c7f-8192-b8809b850eec None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:16.713941 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a1316-a6bd-4c7f-8192-b8809b850eec None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:16.726142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:16.751972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed567503-8e7c-47be-9248-b07fcfaa5037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:16.752691 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed567503-8e7c-47be-9248-b07fcfaa5037 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:16.753015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed567503-8e7c-47be-9248-b07fcfaa5037 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:16.753293 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed567503-8e7c-47be-9248-b07fcfaa5037 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:16.753952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed567503-8e7c-47be-9248-b07fcfaa5037 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:16.755332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed567503-8e7c-47be-9248-b07fcfaa5037 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:16.767626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-ed567503-8e7c-47be-9248-b07fcfaa5037 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 21:56:16.767626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:16.767626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:56:16.767626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:56:16.767626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:56:16.767626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:56:16.767626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:56:16.767626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:56:16.767626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 21:56:16.767626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:56:16.767626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:56:16.767626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:56:16.767626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:16.767626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:16.767626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:16.767626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:16.767626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:16.768822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:16.768822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:16.768822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:16.768822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:16.768822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:16.768822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:16.768822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:56:16.768822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:56:16.768822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:56:16.768822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:56:16.768822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:16.768822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:16.768822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:16.768822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:16.768822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:16.768822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:16.769871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:16.769871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:16.769871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:16.769871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:16.769871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:16.769871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:16.769871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:16.769871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:16.769871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:16.769871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:16.769871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:56:16.769871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:56:16.769871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 21:56:16.769871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:56:16.769871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 651/1290] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Jul 12 21:56:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:16.774765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e419a8f-673d-43ed-922b-1fb4627605f1 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:16.872277 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e419a8f-673d-43ed-922b-1fb4627605f1 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:16.908368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db343c7d-847d-497c-9fee-90311e087fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:16.909067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db343c7d-847d-497c-9fee-90311e087fbc None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:16.909353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db343c7d-847d-497c-9fee-90311e087fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:16.909632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db343c7d-847d-497c-9fee-90311e087fbc None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:16.910377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db343c7d-847d-497c-9fee-90311e087fbc None admin] RBAC: Adding query filter params (name=placement) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:16.910692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db343c7d-847d-497c-9fee-90311e087fbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:16.912077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db343c7d-847d-497c-9fee-90311e087fbc None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:16.917000 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 640/1291] 10.222.0.44 () {56 vars in 1116 bytes} [Tue Jul 12 21:56:16 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:16.923580 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed567503-8e7c-47be-9248-b07fcfaa5037 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:17.022873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed567503-8e7c-47be-9248-b07fcfaa5037 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:17.056093 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402e09e-9b02-4cc4-be6c-563590356fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:17.056764 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402e09e-9b02-4cc4-be6c-563590356fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:17.057047 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402e09e-9b02-4cc4-be6c-563590356fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:17.057321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402e09e-9b02-4cc4-be6c-563590356fe6 None admin] PATH_INFO: `/v3/services` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:17.058017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1402e09e-9b02-4cc4-be6c-563590356fe6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:17.058323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1402e09e-9b02-4cc4-be6c-563590356fe6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:17.059692 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1402e09e-9b02-4cc4-be6c-563590356fe6 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:17.064101 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 652/1292] 10.222.0.44 () {56 vars in 1116 bytes} [Tue Jul 12 21:56:16 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:19.072723 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76d98e-d73f-4a19-b22f-8e9c7ae86ea0 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:19.073245 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76d98e-d73f-4a19-b22f-8e9c7ae86ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:19.073365 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76d98e-d73f-4a19-b22f-8e9c7ae86ea0 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:19.074681 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 641/1293] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:56:19.080155 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd1324-1b4b-4cf8-a41e-6f4106fa8e6b None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:19.080584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd1324-1b4b-4cf8-a41e-6f4106fa8e6b None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:19.080742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd1324-1b4b-4cf8-a41e-6f4106fa8e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:19.097535 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-24cd1324-1b4b-4cf8-a41e-6f4106fa8e6b None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:56:19.098718 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-24cd1324-1b4b-4cf8-a41e-6f4106fa8e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:56:19.128610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:19.165865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 653/1294] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:56:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:56:19.188745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76d98e-d73f-4a19-b22f-8e9c7ae86ea0 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:19.220271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:19.289404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76d98e-d73f-4a19-b22f-8e9c7ae86ea0 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:19.323525 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258c7ad-f5c6-495e-82f5-8358f9483067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:19.324180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258c7ad-f5c6-495e-82f5-8358f9483067 None admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:19.324464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258c7ad-f5c6-495e-82f5-8358f9483067 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:19.324764 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258c7ad-f5c6-495e-82f5-8358f9483067 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:19.325485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258c7ad-f5c6-495e-82f5-8358f9483067 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:19.326854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258c7ad-f5c6-495e-82f5-8358f9483067 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:19.339680 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 642/1295] 10.222.0.44 () {60 vars in 1141 bytes} [Tue Jul 12 21:56:19 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 21:56:21.383135 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec0309-72d0-4a0e-82ac-a9fb28695291 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:21.383662 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec0309-72d0-4a0e-82ac-a9fb28695291 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:21.383775 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec0309-72d0-4a0e-82ac-a9fb28695291 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:21.385268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 654/1296] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:56:21.390265 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84795af-478d-4bea-b326-33ed89e1f073 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:21.390713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84795af-478d-4bea-b326-33ed89e1f073 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:21.390827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84795af-478d-4bea-b326-33ed89e1f073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:21.407807 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-e84795af-478d-4bea-b326-33ed89e1f073 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:56:21.409010 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e84795af-478d-4bea-b326-33ed89e1f073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:56:21.480176 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 643/1297] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:56:21 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:56:21.503018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec0309-72d0-4a0e-82ac-a9fb28695291 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:21.602523 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec0309-72d0-4a0e-82ac-a9fb28695291 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:21.636433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2f906-9014-4860-95e5-782d4b4ecdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:21.637180 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2f906-9014-4860-95e5-782d4b4ecdf8 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:21.637463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2f906-9014-4860-95e5-782d4b4ecdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:21.637738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2f906-9014-4860-95e5-782d4b4ecdf8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:21.638387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2f906-9014-4860-95e5-782d4b4ecdf8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:21.639757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2f906-9014-4860-95e5-782d4b4ecdf8 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:21.655068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-4ca2f906-9014-4860-95e5-782d4b4ecdf8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 21:56:21.655068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:21.655068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:56:21.655068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:56:21.655068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:56:21.655068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:56:21.655068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:56:21.655068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:56:21.655068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 21:56:21.655068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:56:21.655068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:56:21.655068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:56:21.655068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:21.655068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:21.655068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:21.655068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:21.655068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:21.656943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:21.656943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:21.656943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:21.656943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:21.656943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:21.656943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:21.656943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:56:21.656943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:56:21.656943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:56:21.656943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:56:21.656943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:21.656943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:21.656943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:21.656943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:21.656943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:21.656943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:21.657852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:21.657852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:21.657852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:21.657852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:21.657852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:21.657852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:21.657852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:21.657852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:21.657852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:21.657852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:21.657852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:56:21.657852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:56:21.657852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 21:56:21.657852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:56:21.657852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 655/1298] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Jul 12 21:56:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:21.665259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84795af-478d-4bea-b326-33ed89e1f073 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:21.691558 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:21.782258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84795af-478d-4bea-b326-33ed89e1f073 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:21.826035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055e117-220b-47c9-95d4-eed67157f12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:21.826701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055e117-220b-47c9-95d4-eed67157f12d None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:21.826990 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055e117-220b-47c9-95d4-eed67157f12d None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:21.827270 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055e117-220b-47c9-95d4-eed67157f12d None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:21.828518 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055e117-220b-47c9-95d4-eed67157f12d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:21.828855 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055e117-220b-47c9-95d4-eed67157f12d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:21.830249 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055e117-220b-47c9-95d4-eed67157f12d None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:21.835652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 644/1299] 10.222.0.44 () {56 vars in 1116 bytes} [Tue Jul 12 21:56:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:21.841695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2f906-9014-4860-95e5-782d4b4ecdf8 None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:21.874418 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:21.944193 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2f906-9014-4860-95e5-782d4b4ecdf8 None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:21.979316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a4200-48eb-4e15-af11-abfe92f63464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:21.979980 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a4200-48eb-4e15-af11-abfe92f63464 None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:21.980268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a4200-48eb-4e15-af11-abfe92f63464 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:21.980552 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a4200-48eb-4e15-af11-abfe92f63464 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:21.981376 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a4200-48eb-4e15-af11-abfe92f63464 None admin] RBAC: Adding query filter params (interface=public, service_id=82fc180bb5894313888dd604b2fc064d, region_id=RegionOne) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:21.981741 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a4200-48eb-4e15-af11-abfe92f63464 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:21.983072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a4200-48eb-4e15-af11-abfe92f63464 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:21.988251 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 656/1300] 10.222.0.44 () {56 vars in 1251 bytes} [Tue Jul 12 21:56:21 2022] GET /identity/v3/endpoints?service_id=82fc180bb5894313888dd604b2fc064d&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:21.993884 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055e117-220b-47c9-95d4-eed67157f12d None admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:22.075942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:22.098321 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055e117-220b-47c9-95d4-eed67157f12d None admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:22.139468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380ac27-c7e8-46ee-978c-390a4cceef76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:22.141143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380ac27-c7e8-46ee-978c-390a4cceef76 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:22.141575 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380ac27-c7e8-46ee-978c-390a4cceef76 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:22.142001 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380ac27-c7e8-46ee-978c-390a4cceef76 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:22.143074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380ac27-c7e8-46ee-978c-390a4cceef76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:22.145288 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380ac27-c7e8-46ee-978c-390a4cceef76 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:22.152900 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 645/1301] 10.222.0.44 () {56 vars in 1087 bytes} [Tue Jul 12 21:56:21 2022] GET /identity/v3/services => generated 2191 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 21:56:24.240191 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cea2e7-9ff9-4ddf-8a1e-fb6f8eb89579 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:24.240727 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cea2e7-9ff9-4ddf-8a1e-fb6f8eb89579 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:24.240727 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cea2e7-9ff9-4ddf-8a1e-fb6f8eb89579 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:24.243035 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 657/1302] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:56:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:56:24.248363 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea71c1-98fe-4f98-9a5a-50fe28668824 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:24.248856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea71c1-98fe-4f98-9a5a-50fe28668824 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:24.248958 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea71c1-98fe-4f98-9a5a-50fe28668824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:24.269767 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-a3ea71c1-98fe-4f98-9a5a-50fe28668824 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:56:24.270949 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a3ea71c1-98fe-4f98-9a5a-50fe28668824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:56:24.337793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 646/1303] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:56:24 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 21:56:24.361133 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cea2e7-9ff9-4ddf-8a1e-fb6f8eb89579 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:24.458727 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:24.466111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cea2e7-9ff9-4ddf-8a1e-fb6f8eb89579 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:24.499649 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b69a24-0401-4fc6-80f7-cfe703a26fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:24.500352 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b69a24-0401-4fc6-80f7-cfe703a26fad None admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:24.500666 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b69a24-0401-4fc6-80f7-cfe703a26fad None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:24.500949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b69a24-0401-4fc6-80f7-cfe703a26fad None admin] PATH_INFO: `/v3/services/placement` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:24.501597 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b69a24-0401-4fc6-80f7-cfe703a26fad None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:24.502971 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b69a24-0401-4fc6-80f7-cfe703a26fad None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:24.515364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application [None req-f7b69a24-0401-4fc6-80f7-cfe703a26fad None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 21:56:24.515364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:24.515364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 21:56:24.515364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 21:56:24.515364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 21:56:24.515364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 21:56:24.515364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 21:56:24.515364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:56:24.515364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 21:56:24.515364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:56:24.515364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 21:56:24.515364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:56:24.515364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 21:56:24.515364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 21:56:24.515364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 21:56:24.515364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 21:56:24.515364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 21:56:24.516753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 21:56:24.516753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 21:56:24.516753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 21:56:24.516753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 21:56:24.516753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 21:56:24.516753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 21:56:24.516753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 21:56:24.516753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 21:56:24.516753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 21:56:24.516753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 21:56:24.516753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 21:56:24.516753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 21:56:24.516753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 21:56:24.516753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 21:56:24.516753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 21:56:24.516753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 21:56:24.517919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 21:56:24.517919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application with Lock( Jul 12 21:56:24.517919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 21:56:24.517919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self._enter() Jul 12 21:56:24.517919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 21:56:24.517919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 21:56:24.517919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 21:56:24.517919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application return self.creator() Jul 12 21:56:24.517919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 21:56:24.517919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application created_value = creator( Jul 12 21:56:24.517919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 21:56:24.517919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 21:56:24.517919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 21:56:24.517919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: ERROR keystone.server.flask.application Jul 12 21:56:24.517919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 658/1304] 10.222.0.44 () {56 vars in 1117 bytes} [Tue Jul 12 21:56:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 21:56:24.522674 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea71c1-98fe-4f98-9a5a-50fe28668824 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:24.601364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:24.625985 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea71c1-98fe-4f98-9a5a-50fe28668824 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:24.658969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec63360-22f5-4975-b995-b1f3b849cd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:24.659616 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec63360-22f5-4975-b995-b1f3b849cd09 None admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:24.659895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec63360-22f5-4975-b995-b1f3b849cd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:24.660165 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec63360-22f5-4975-b995-b1f3b849cd09 None admin] PATH_INFO: `/v3/services` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:24.660889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec63360-22f5-4975-b995-b1f3b849cd09 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 21:56:24.661198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec63360-22f5-4975-b995-b1f3b849cd09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:24.662560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec63360-22f5-4975-b995-b1f3b849cd09 None admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:24.667529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 647/1305] 10.222.0.44 () {56 vars in 1116 bytes} [Tue Jul 12 21:56:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 21:56:24.674208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b69a24-0401-4fc6-80f7-cfe703a26fad None admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:56:24.771349 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b69a24-0401-4fc6-80f7-cfe703a26fad None admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:56:24.803637 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ddbc7-e676-425a-8eba-3b9579fb1ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:56:24.804274 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ddbc7-e676-425a-8eba-3b9579fb1ed2 None admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:56:24.804556 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ddbc7-e676-425a-8eba-3b9579fb1ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:56:24.804861 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ddbc7-e676-425a-8eba-3b9579fb1ed2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:56:24.805599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ddbc7-e676-425a-8eba-3b9579fb1ed2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:56:24.806949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ddbc7-e676-425a-8eba-3b9579fb1ed2 None admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:56:24.824755 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 659/1306] 10.222.0.44 () {60 vars in 1144 bytes} [Tue Jul 12 21:56:24 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 21:56:34.664755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=30 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:56:34.820529 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:28.749305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0903fc-e912-47e4-96f3-543fe711fafc None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:28.749886 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0903fc-e912-47e4-96f3-543fe711fafc None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:28.750176 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0903fc-e912-47e4-96f3-543fe711fafc None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:28.752594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 648/1307] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:57:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:57:28.758516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ee3ae-f1ce-4e0e-ae00-b5263a3dbc2f None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:28.759014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ee3ae-f1ce-4e0e-ae00-b5263a3dbc2f None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:28.759096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ee3ae-f1ce-4e0e-ae00-b5263a3dbc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:28.858997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-856ee3ae-f1ce-4e0e-ae00-b5263a3dbc2f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:57:28.860258 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-856ee3ae-f1ce-4e0e-ae00-b5263a3dbc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:29.009356 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 660/1308] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:57:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:57:29.041979 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ee46b-cdcc-44ec-b687-3981184899b6 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:29.042266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ee46b-cdcc-44ec-b687-3981184899b6 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:29.042551 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ee46b-cdcc-44ec-b687-3981184899b6 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:29.044878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 649/1309] 10.222.0.44 () {54 vars in 964 bytes} [Tue Jul 12 21:57:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:57:29.051338 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d108e-e9b1-4d41-ae37-17905cfc2a26 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:29.051639 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d108e-e9b1-4d41-ae37-17905cfc2a26 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:29.051924 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d108e-e9b1-4d41-ae37-17905cfc2a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:29.137783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-6c4d108e-e9b1-4d41-ae37-17905cfc2a26 None None] MFA Rules not processed for user `53b891a734a64b5983564e207de0359d`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:57:29.139062 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-6c4d108e-e9b1-4d41-ae37-17905cfc2a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:29.162687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:29.229861 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 661/1310] 10.222.0.44 () {58 vars in 1061 bytes} [Tue Jul 12 21:57:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:57:29.240210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ee46b-cdcc-44ec-b687-3981184899b6 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:57:29.242081 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-5f9ee46b-cdcc-44ec-b687-3981184899b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:29.356002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ee46b-cdcc-44ec-b687-3981184899b6 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:57:29.436453 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75205d1-fca0-4709-b8a5-9c732b6c453d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:57:29.437154 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75205d1-fca0-4709-b8a5-9c732b6c453d service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:29.437433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75205d1-fca0-4709-b8a5-9c732b6c453d service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:29.437709 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75205d1-fca0-4709-b8a5-9c732b6c453d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:29.439384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d75205d1-fca0-4709-b8a5-9c732b6c453d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:29.505501 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:29.602936 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75205d1-fca0-4709-b8a5-9c732b6c453d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:57:29.606080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75205d1-fca0-4709-b8a5-9c732b6c453d service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:57:29.635130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 650/1311] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:57:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:57:32.003105 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db33dc3-898c-4f15-aa2e-e35ec19c9ae7 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:32.003751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db33dc3-898c-4f15-aa2e-e35ec19c9ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:32.003751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db33dc3-898c-4f15-aa2e-e35ec19c9ae7 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:32.005085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 662/1312] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:57:32.010826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143be7b4-1c1e-4dba-a288-03126434ac19 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:32.012153 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143be7b4-1c1e-4dba-a288-03126434ac19 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:32.012434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143be7b4-1c1e-4dba-a288-03126434ac19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:32.094540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-143be7b4-1c1e-4dba-a288-03126434ac19 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:57:32.095798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-143be7b4-1c1e-4dba-a288-03126434ac19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:32.208486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 651/1313] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:57:32.245954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db33dc3-898c-4f15-aa2e-e35ec19c9ae7 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:57:32.247651 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-9db33dc3-898c-4f15-aa2e-e35ec19c9ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:32.365879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db33dc3-898c-4f15-aa2e-e35ec19c9ae7 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:57:32.447279 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:32.450145 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b390790-1817-4023-b0c5-2226d7df20a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:57:32.450894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b390790-1817-4023-b0c5-2226d7df20a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:32.451427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b390790-1817-4023-b0c5-2226d7df20a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:32.451942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b390790-1817-4023-b0c5-2226d7df20a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:32.453960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2b390790-1817-4023-b0c5-2226d7df20a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:32.564376 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b390790-1817-4023-b0c5-2226d7df20a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:57:32.566133 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b390790-1817-4023-b0c5-2226d7df20a7 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:57:32.586063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 663/1314] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:57:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:57:34.986058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e326b10-db97-413c-bd1e-fa45068b43d5 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:34.986516 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e326b10-db97-413c-bd1e-fa45068b43d5 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:34.986622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e326b10-db97-413c-bd1e-fa45068b43d5 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:34.987998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 652/1315] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:57:34.993478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cf0fa-5cd8-4169-8192-07874171d345 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:34.993931 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cf0fa-5cd8-4169-8192-07874171d345 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:34.994047 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cf0fa-5cd8-4169-8192-07874171d345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:35.057841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-0a3cf0fa-5cd8-4169-8192-07874171d345 None None] MFA Rules not processed for user `f0a92aa0e8164ee0890c4bb9a039faf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:57:35.059094 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0a3cf0fa-5cd8-4169-8192-07874171d345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:35.110451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:35.155586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 664/1316] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:57:34 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:57:35.199265 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e326b10-db97-413c-bd1e-fa45068b43d5 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:57:35.200934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6e326b10-db97-413c-bd1e-fa45068b43d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:35.210593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:35.306753 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e326b10-db97-413c-bd1e-fa45068b43d5 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:57:35.377986 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f863a63-53ef-4f97-865d-b3e18f107906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:57:35.378625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f863a63-53ef-4f97-865d-b3e18f107906 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:35.378906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f863a63-53ef-4f97-865d-b3e18f107906 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:35.379180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f863a63-53ef-4f97-865d-b3e18f107906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:35.380841 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0f863a63-53ef-4f97-865d-b3e18f107906 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:35.441773 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:35.488628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f863a63-53ef-4f97-865d-b3e18f107906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:57:35.490106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f863a63-53ef-4f97-865d-b3e18f107906 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:57:35.508472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 653/1317] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:57:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:57:38.120355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d36108-6931-4097-9065-ef92a52e42b8 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:38.121805 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d36108-6931-4097-9065-ef92a52e42b8 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:38.121805 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d36108-6931-4097-9065-ef92a52e42b8 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:38.123091 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 665/1318] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:57:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:57:38.128384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a283571-9775-4aff-bb72-05a4ccce4cce None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:38.128799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a283571-9775-4aff-bb72-05a4ccce4cce None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:38.128989 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a283571-9775-4aff-bb72-05a4ccce4cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:38.192586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-5a283571-9775-4aff-bb72-05a4ccce4cce None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:57:38.193824 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-5a283571-9775-4aff-bb72-05a4ccce4cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:38.289598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 654/1319] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:57:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:57:38.342389 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd256494-d143-446b-a473-eb6875de0db3 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:38.342737 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd256494-d143-446b-a473-eb6875de0db3 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:38.342965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd256494-d143-446b-a473-eb6875de0db3 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:38.344388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 666/1320] 10.222.0.44 () {54 vars in 964 bytes} [Tue Jul 12 21:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:57:38.351350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4d80a-6fad-40e7-86cd-7808ccf30e59 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:38.351350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4d80a-6fad-40e7-86cd-7808ccf30e59 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:38.351350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4d80a-6fad-40e7-86cd-7808ccf30e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:38.375125 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:38.432029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-9ec4d80a-6fad-40e7-86cd-7808ccf30e59 None None] MFA Rules not processed for user `53b891a734a64b5983564e207de0359d`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:57:38.433527 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-9ec4d80a-6fad-40e7-86cd-7808ccf30e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:38.509135 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 655/1321] 10.222.0.44 () {58 vars in 1061 bytes} [Tue Jul 12 21:57:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:57:38.518501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd256494-d143-446b-a473-eb6875de0db3 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:57:38.520028 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-dd256494-d143-446b-a473-eb6875de0db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:38.622358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd256494-d143-446b-a473-eb6875de0db3 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:57:38.704059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768af5bc-aa7f-47dd-a813-ee3da82491e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:57:38.704749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768af5bc-aa7f-47dd-a813-ee3da82491e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:38.705038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768af5bc-aa7f-47dd-a813-ee3da82491e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:38.705319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768af5bc-aa7f-47dd-a813-ee3da82491e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:38.707009 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-768af5bc-aa7f-47dd-a813-ee3da82491e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:38.715062 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:38.817011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768af5bc-aa7f-47dd-a813-ee3da82491e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:57:38.818453 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768af5bc-aa7f-47dd-a813-ee3da82491e6 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:57:38.837901 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 667/1322] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:57:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:57:42.140092 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a392f0-b809-4ef8-9e8f-12013bc0bd28 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:42.140704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a392f0-b809-4ef8-9e8f-12013bc0bd28 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:42.140704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a392f0-b809-4ef8-9e8f-12013bc0bd28 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:42.142051 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 656/1323] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:57:42.147223 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eceae4-6d08-4738-a493-e01fc903f628 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:42.147684 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eceae4-6d08-4738-a493-e01fc903f628 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:42.147792 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eceae4-6d08-4738-a493-e01fc903f628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:42.213833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-e2eceae4-6d08-4738-a493-e01fc903f628 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:57:42.215240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e2eceae4-6d08-4738-a493-e01fc903f628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:42.275127 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:42.329256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 668/1324] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:57:42.367200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a392f0-b809-4ef8-9e8f-12013bc0bd28 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:57:42.368377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-12a392f0-b809-4ef8-9e8f-12013bc0bd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:42.483006 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a392f0-b809-4ef8-9e8f-12013bc0bd28 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:57:42.496888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:42.563168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d162f-01c1-4b24-b2ea-e112eb6740af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:57:42.564177 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d162f-01c1-4b24-b2ea-e112eb6740af service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:42.564956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d162f-01c1-4b24-b2ea-e112eb6740af service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:42.565444 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d162f-01c1-4b24-b2ea-e112eb6740af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:42.568595 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b38d162f-01c1-4b24-b2ea-e112eb6740af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:42.691992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d162f-01c1-4b24-b2ea-e112eb6740af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:57:42.693596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d162f-01c1-4b24-b2ea-e112eb6740af service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:57:42.713715 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 657/1325] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:57:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:57:44.841846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678ae80-3e58-49e4-9c5a-c65d077f2880 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:44.842520 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678ae80-3e58-49e4-9c5a-c65d077f2880 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:44.842520 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678ae80-3e58-49e4-9c5a-c65d077f2880 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:44.843719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 669/1326] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:57:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:57:44.849243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a3f4c-93b3-4938-bd96-c7a97ae4481b None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:44.849707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a3f4c-93b3-4938-bd96-c7a97ae4481b None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:44.849809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a3f4c-93b3-4938-bd96-c7a97ae4481b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:44.901004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:44.920939 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-900a3f4c-93b3-4938-bd96-c7a97ae4481b None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:57:44.922165 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-900a3f4c-93b3-4938-bd96-c7a97ae4481b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:45.043635 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 658/1327] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:57:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:57:45.083968 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678ae80-3e58-49e4-9c5a-c65d077f2880 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:57:45.085581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-f678ae80-3e58-49e4-9c5a-c65d077f2880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:45.194788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678ae80-3e58-49e4-9c5a-c65d077f2880 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:57:45.275161 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9bcb0-c41d-4e17-9a13-21d403cb00a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:57:45.275817 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9bcb0-c41d-4e17-9a13-21d403cb00a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:45.276110 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9bcb0-c41d-4e17-9a13-21d403cb00a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:45.276653 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9bcb0-c41d-4e17-9a13-21d403cb00a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:45.278292 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5ba9bcb0-c41d-4e17-9a13-21d403cb00a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:45.297806 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:45.405495 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9bcb0-c41d-4e17-9a13-21d403cb00a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:57:45.406972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9bcb0-c41d-4e17-9a13-21d403cb00a1 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:57:45.427038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 670/1328] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:57:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:57:47.570357 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bb864-f70b-4765-ae9b-a2b7075cb24e None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:47.570912 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bb864-f70b-4765-ae9b-a2b7075cb24e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:47.570912 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bb864-f70b-4765-ae9b-a2b7075cb24e None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:47.572282 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 659/1329] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:57:47.578894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67784a-ee63-4c09-90a7-b608497772e4 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:47.579441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67784a-ee63-4c09-90a7-b608497772e4 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:47.579441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67784a-ee63-4c09-90a7-b608497772e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:47.657413 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-9c67784a-ee63-4c09-90a7-b608497772e4 None None] MFA Rules not processed for user `f0a92aa0e8164ee0890c4bb9a039faf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:57:47.658609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-9c67784a-ee63-4c09-90a7-b608497772e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:47.726787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:47.761526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 671/1330] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:57:47 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:57:47.798346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085bb864-f70b-4765-ae9b-a2b7075cb24e None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:57:47.799898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-085bb864-f70b-4765-ae9b-a2b7075cb24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:47.936536 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085bb864-f70b-4765-ae9b-a2b7075cb24e None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:57:47.968674 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:48.024559 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07196e6-dfa6-4e1a-9368-e170678c55f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:57:48.025238 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07196e6-dfa6-4e1a-9368-e170678c55f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:48.025527 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07196e6-dfa6-4e1a-9368-e170678c55f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:48.025804 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07196e6-dfa6-4e1a-9368-e170678c55f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:48.028280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-f07196e6-dfa6-4e1a-9368-e170678c55f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:48.162180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07196e6-dfa6-4e1a-9368-e170678c55f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:57:48.163666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07196e6-dfa6-4e1a-9368-e170678c55f6 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:57:48.185588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 660/1331] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:57:47 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:57:50.519172 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce64f6-ffba-429b-9246-4081b4c213de None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:50.519743 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce64f6-ffba-429b-9246-4081b4c213de None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:50.519743 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce64f6-ffba-429b-9246-4081b4c213de None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:50.521140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 672/1332] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:57:50.526924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da549af-785e-49ea-8ca9-49e50b860f6f None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:50.527493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da549af-785e-49ea-8ca9-49e50b860f6f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:50.527493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da549af-785e-49ea-8ca9-49e50b860f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:50.592946 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:50.596464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-2da549af-785e-49ea-8ca9-49e50b860f6f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:57:50.597886 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-2da549af-785e-49ea-8ca9-49e50b860f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:50.694390 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 661/1333] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:57:50.730870 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce64f6-ffba-429b-9246-4081b4c213de None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:57:50.732430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-87ce64f6-ffba-429b-9246-4081b4c213de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:50.833962 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce64f6-ffba-429b-9246-4081b4c213de None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:57:50.910667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd6f00-a014-4478-88b2-e1ce1575bc7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:57:50.911318 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd6f00-a014-4478-88b2-e1ce1575bc7b service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:50.911607 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd6f00-a014-4478-88b2-e1ce1575bc7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:50.911893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd6f00-a014-4478-88b2-e1ce1575bc7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:50.913579 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-85cd6f00-a014-4478-88b2-e1ce1575bc7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:50.941540 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:51.024638 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd6f00-a014-4478-88b2-e1ce1575bc7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:57:51.026821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd6f00-a014-4478-88b2-e1ce1575bc7b service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:57:51.045460 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 673/1334] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:57:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:57:53.142192 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca1dd3-42d2-436f-9828-83612d1e7b36 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:53.142751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca1dd3-42d2-436f-9828-83612d1e7b36 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:53.142848 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca1dd3-42d2-436f-9828-83612d1e7b36 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:53.144175 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 662/1335] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:57:53.149426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb6dea-e6dc-4341-868d-1c40b51c254c None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:53.149868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb6dea-e6dc-4341-868d-1c40b51c254c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:53.149999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb6dea-e6dc-4341-868d-1c40b51c254c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:53.215438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-fbeb6dea-e6dc-4341-868d-1c40b51c254c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:57:53.216595 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-fbeb6dea-e6dc-4341-868d-1c40b51c254c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:53.290478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:53.314960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 674/1336] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:57:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:57:53.356109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca1dd3-42d2-436f-9828-83612d1e7b36 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:57:53.358333 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b5ca1dd3-42d2-436f-9828-83612d1e7b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:53.474811 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca1dd3-42d2-436f-9828-83612d1e7b36 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:57:53.499023 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:53.545889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc951f88-53ee-4acc-b75d-26b257248980 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:57:53.546543 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc951f88-53ee-4acc-b75d-26b257248980 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:53.546834 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc951f88-53ee-4acc-b75d-26b257248980 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:53.547116 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc951f88-53ee-4acc-b75d-26b257248980 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:53.548816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-cc951f88-53ee-4acc-b75d-26b257248980 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:53.658335 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc951f88-53ee-4acc-b75d-26b257248980 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:57:53.659786 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc951f88-53ee-4acc-b75d-26b257248980 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:57:53.678470 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 663/1337] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:57:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:57:56.118842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298abc17-0514-4b9c-b0bf-85984e3bf41b None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:56.119430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298abc17-0514-4b9c-b0bf-85984e3bf41b None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:56.120300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298abc17-0514-4b9c-b0bf-85984e3bf41b None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:56.122738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 675/1338] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:57:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:57:56.128521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa81c5-178a-4567-9271-8b7e1127a606 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:56.129056 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa81c5-178a-4567-9271-8b7e1127a606 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:56.129158 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa81c5-178a-4567-9271-8b7e1127a606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:56.211761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-67fa81c5-178a-4567-9271-8b7e1127a606 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:57:56.213054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-67fa81c5-178a-4567-9271-8b7e1127a606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:56.221570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:56.310715 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 664/1339] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:57:56 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:57:56.347061 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298abc17-0514-4b9c-b0bf-85984e3bf41b None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:57:56.348660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-298abc17-0514-4b9c-b0bf-85984e3bf41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:56.460407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298abc17-0514-4b9c-b0bf-85984e3bf41b None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:57:56.538571 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb07ff-f2b0-4a06-940c-9542f5ddc996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:57:56.539222 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb07ff-f2b0-4a06-940c-9542f5ddc996 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:57:56.539510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb07ff-f2b0-4a06-940c-9542f5ddc996 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:57:56.539792 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb07ff-f2b0-4a06-940c-9542f5ddc996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:57:56.541524 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-6ecb07ff-f2b0-4a06-940c-9542f5ddc996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:57:56.576888 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:57:56.650038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb07ff-f2b0-4a06-940c-9542f5ddc996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:57:56.652303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb07ff-f2b0-4a06-940c-9542f5ddc996 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:57:56.671491 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 676/1340] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:57:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:58:00.688972 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b389c-3695-4b91-858e-6a00771aeec3 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:00.689802 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b389c-3695-4b91-858e-6a00771aeec3 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:00.689802 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b389c-3695-4b91-858e-6a00771aeec3 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:00.691089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 665/1341] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:00.696641 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ec948-6e4c-46fd-98c0-77d592b16ba5 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:00.697102 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ec948-6e4c-46fd-98c0-77d592b16ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:00.697237 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ec948-6e4c-46fd-98c0-77d592b16ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:00.774210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-704ec948-6e4c-46fd-98c0-77d592b16ba5 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:00.775592 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-704ec948-6e4c-46fd-98c0-77d592b16ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:00.867652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:00.881818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 677/1342] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:58:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:00.922924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b389c-3695-4b91-858e-6a00771aeec3 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:00.924637 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-523b389c-3695-4b91-858e-6a00771aeec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:01.036566 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b389c-3695-4b91-858e-6a00771aeec3 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:01.079199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:01.116371 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156211bd-d02d-4caf-b9e2-29a7753b98e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:01.117063 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156211bd-d02d-4caf-b9e2-29a7753b98e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:01.117348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156211bd-d02d-4caf-b9e2-29a7753b98e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:01.117635 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156211bd-d02d-4caf-b9e2-29a7753b98e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:01.119305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-156211bd-d02d-4caf-b9e2-29a7753b98e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:01.234513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156211bd-d02d-4caf-b9e2-29a7753b98e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:01.235884 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156211bd-d02d-4caf-b9e2-29a7753b98e2 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:01.255327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 666/1343] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:58:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:58:04.058690 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae6ba6-a23e-4c70-8b52-ccddf413128f None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:04.059536 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae6ba6-a23e-4c70-8b52-ccddf413128f None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:04.059699 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae6ba6-a23e-4c70-8b52-ccddf413128f None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:04.061203 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 678/1344] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:58:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:04.066764 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fec6f-dce5-4949-9e23-88d86be48147 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:04.067208 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fec6f-dce5-4949-9e23-88d86be48147 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:04.067337 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fec6f-dce5-4949-9e23-88d86be48147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:04.132180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-231fec6f-dce5-4949-9e23-88d86be48147 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:04.133409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-231fec6f-dce5-4949-9e23-88d86be48147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:04.152307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:04.233922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 667/1345] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:58:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:04.272121 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae6ba6-a23e-4c70-8b52-ccddf413128f None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:04.273784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2eae6ba6-a23e-4c70-8b52-ccddf413128f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:04.378604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae6ba6-a23e-4c70-8b52-ccddf413128f None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:04.452769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3370f-fcc1-4ed1-9027-e8b85f0a6af3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:04.453420 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3370f-fcc1-4ed1-9027-e8b85f0a6af3 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:04.453702 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3370f-fcc1-4ed1-9027-e8b85f0a6af3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:04.453984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3370f-fcc1-4ed1-9027-e8b85f0a6af3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:04.455618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-7dc3370f-fcc1-4ed1-9027-e8b85f0a6af3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:04.498956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:04.565367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3370f-fcc1-4ed1-9027-e8b85f0a6af3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:04.566811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3370f-fcc1-4ed1-9027-e8b85f0a6af3 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:04.584803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 679/1346] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:58:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:58:08.942910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7482b0f-2656-43ec-865e-bf1d45c99f72 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:08.943469 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7482b0f-2656-43ec-865e-bf1d45c99f72 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:08.943469 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7482b0f-2656-43ec-865e-bf1d45c99f72 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:08.944880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 668/1347] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:08.951046 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac85e7-b9a1-473d-aecf-63921ff313d0 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:08.951643 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac85e7-b9a1-473d-aecf-63921ff313d0 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:08.951643 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac85e7-b9a1-473d-aecf-63921ff313d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:09.024347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-cfac85e7-b9a1-473d-aecf-63921ff313d0 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:09.025826 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-cfac85e7-b9a1-473d-aecf-63921ff313d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:09.132151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:09.137258 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 680/1348] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:09.173624 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7482b0f-2656-43ec-865e-bf1d45c99f72 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:09.175207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-f7482b0f-2656-43ec-865e-bf1d45c99f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:09.282104 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7482b0f-2656-43ec-865e-bf1d45c99f72 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:09.343387 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:09.375093 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f3ff9-9fb4-46cf-b033-7487c9fea0ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:09.375743 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f3ff9-9fb4-46cf-b033-7487c9fea0ab service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:09.376034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f3ff9-9fb4-46cf-b033-7487c9fea0ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:09.376318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f3ff9-9fb4-46cf-b033-7487c9fea0ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:09.378037 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-085f3ff9-9fb4-46cf-b033-7487c9fea0ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:09.489323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f3ff9-9fb4-46cf-b033-7487c9fea0ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:09.490684 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f3ff9-9fb4-46cf-b033-7487c9fea0ab service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:09.509143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 669/1349] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:58:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:58:14.566535 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43cdf0-57ad-4163-a40e-a3cda49de7c3 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:14.567039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43cdf0-57ad-4163-a40e-a3cda49de7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:14.567133 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43cdf0-57ad-4163-a40e-a3cda49de7c3 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:14.568745 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 681/1350] 10.222.0.44 () {54 vars in 921 bytes} [Tue Jul 12 21:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:14.576124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e9bd4-41e5-4ae0-83f7-bcab92f34244 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:14.576962 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e9bd4-41e5-4ae0-83f7-bcab92f34244 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:14.577470 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e9bd4-41e5-4ae0-83f7-bcab92f34244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:14.715578 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-347e9bd4-41e5-4ae0-83f7-bcab92f34244 None None] MFA Rules not processed for user `a1f1ef52f8eb42a59a50a69873b9ffaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:14.724816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-347e9bd4-41e5-4ae0-83f7-bcab92f34244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:14.750502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:14.863551 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 670/1351] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Jul 12 21:58:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:19.034929 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f41da-ef3b-4c1e-a1b2-919f5fe10cef None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:19.035823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f41da-ef3b-4c1e-a1b2-919f5fe10cef None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:19.036106 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f41da-ef3b-4c1e-a1b2-919f5fe10cef None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:19.037520 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 682/1352] 10.222.0.44 () {54 vars in 921 bytes} [Tue Jul 12 21:58:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:19.042770 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2fbe-abff-4c58-b3be-c399e0c16b7b None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:19.045001 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2fbe-abff-4c58-b3be-c399e0c16b7b None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:19.045302 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2fbe-abff-4c58-b3be-c399e0c16b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:19.126458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-6f9c2fbe-abff-4c58-b3be-c399e0c16b7b None None] MFA Rules not processed for user `a1f1ef52f8eb42a59a50a69873b9ffaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:19.127670 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6f9c2fbe-abff-4c58-b3be-c399e0c16b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:19.204007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:19.246972 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 671/1353] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Jul 12 21:58:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:20.732509 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e724f2-cbe4-4f36-9824-06cc60aab335 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:20.733052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e724f2-cbe4-4f36-9824-06cc60aab335 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:20.733052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e724f2-cbe4-4f36-9824-06cc60aab335 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:20.734451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 683/1354] 10.222.0.44 () {54 vars in 921 bytes} [Tue Jul 12 21:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:20.739805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067f8d7-1a52-4ca1-ad1c-42e8454b3567 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:20.740269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067f8d7-1a52-4ca1-ad1c-42e8454b3567 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:20.740511 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067f8d7-1a52-4ca1-ad1c-42e8454b3567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:20.818832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-3067f8d7-1a52-4ca1-ad1c-42e8454b3567 None None] MFA Rules not processed for user `a1f1ef52f8eb42a59a50a69873b9ffaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:20.820047 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-3067f8d7-1a52-4ca1-ad1c-42e8454b3567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:20.942936 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 672/1355] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Jul 12 21:58:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:24.273410 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6b277-36c4-4b8f-a241-1990d79db346 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:24.273962 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6b277-36c4-4b8f-a241-1990d79db346 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:24.273962 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6b277-36c4-4b8f-a241-1990d79db346 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:24.275374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 684/1356] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:24.280624 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b756776-8875-4269-9bdc-599d7b5cd20f None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:24.281283 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b756776-8875-4269-9bdc-599d7b5cd20f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:24.281379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b756776-8875-4269-9bdc-599d7b5cd20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:24.329457 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:24.371220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-2b756776-8875-4269-9bdc-599d7b5cd20f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:24.372960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-2b756776-8875-4269-9bdc-599d7b5cd20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:24.501262 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 673/1357] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:24.558894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab58361-7265-473e-b4df-86dd28a8efb6 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:24.559189 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab58361-7265-473e-b4df-86dd28a8efb6 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:24.559470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab58361-7265-473e-b4df-86dd28a8efb6 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:24.561355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 685/1358] 10.222.0.44 () {54 vars in 961 bytes} [Tue Jul 12 21:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:24.574190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c25485-67bb-47b6-b61d-91ec6497da93 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:24.574698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c25485-67bb-47b6-b61d-91ec6497da93 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:24.575072 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c25485-67bb-47b6-b61d-91ec6497da93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:24.661893 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-f9c25485-67bb-47b6-b61d-91ec6497da93 None None] MFA Rules not processed for user `24915625354f418eb7bc4d5ada85fdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:24.663241 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-f9c25485-67bb-47b6-b61d-91ec6497da93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:24.721428 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:24.808433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 674/1359] 10.222.0.44 () {58 vars in 1058 bytes} [Tue Jul 12 21:58:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 21:58:24.817381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab58361-7265-473e-b4df-86dd28a8efb6 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:24.819369 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0ab58361-7265-473e-b4df-86dd28a8efb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:24.980859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab58361-7265-473e-b4df-86dd28a8efb6 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:25.078716 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:25.129820 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592d949-c281-4f07-a7f9-f4a2319dfc7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:25.130713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592d949-c281-4f07-a7f9-f4a2319dfc7f service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:25.131106 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592d949-c281-4f07-a7f9-f4a2319dfc7f service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:25.131484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592d949-c281-4f07-a7f9-f4a2319dfc7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:25.133271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8592d949-c281-4f07-a7f9-f4a2319dfc7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:25.234868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592d949-c281-4f07-a7f9-f4a2319dfc7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:25.236635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592d949-c281-4f07-a7f9-f4a2319dfc7f service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:25.257394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 686/1360] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:58:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:58:27.576233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f600ad9-5afb-4d1d-88bc-e02429be9a57 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:27.576630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f600ad9-5afb-4d1d-88bc-e02429be9a57 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:27.576630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f600ad9-5afb-4d1d-88bc-e02429be9a57 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:27.578156 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 675/1361] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:27.583440 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe14ef2-8d94-4f9e-ac37-92a46b7ceca7 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:27.583974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe14ef2-8d94-4f9e-ac37-92a46b7ceca7 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:27.583974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe14ef2-8d94-4f9e-ac37-92a46b7ceca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:27.656582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-2fe14ef2-8d94-4f9e-ac37-92a46b7ceca7 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:27.657782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2fe14ef2-8d94-4f9e-ac37-92a46b7ceca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:27.708798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:27.778094 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 687/1362] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:58:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:27.822648 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f600ad9-5afb-4d1d-88bc-e02429be9a57 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:27.824178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4f600ad9-5afb-4d1d-88bc-e02429be9a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:28.032353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f600ad9-5afb-4d1d-88bc-e02429be9a57 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:28.045145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:28.210554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e3a00-d518-4cf1-8b6f-ba7b2d4d4ac4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:28.211309 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e3a00-d518-4cf1-8b6f-ba7b2d4d4ac4 service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:28.211677 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e3a00-d518-4cf1-8b6f-ba7b2d4d4ac4 service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:28.212593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e3a00-d518-4cf1-8b6f-ba7b2d4d4ac4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:28.214426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e74e3a00-d518-4cf1-8b6f-ba7b2d4d4ac4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:28.312315 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e3a00-d518-4cf1-8b6f-ba7b2d4d4ac4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:28.313955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e3a00-d518-4cf1-8b6f-ba7b2d4d4ac4 service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:28.333835 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 676/1363] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:58:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:58:30.796243 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476be3c-0f17-41f3-8dd1-b4c93d1eafd4 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:30.797145 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476be3c-0f17-41f3-8dd1-b4c93d1eafd4 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:30.797430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476be3c-0f17-41f3-8dd1-b4c93d1eafd4 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:30.798845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 688/1364] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:58:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:30.804527 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb99a22-49c8-4717-9f3b-170959979bfb None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:30.805061 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb99a22-49c8-4717-9f3b-170959979bfb None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:30.805883 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb99a22-49c8-4717-9f3b-170959979bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:30.826079 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:30.905294 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-ffb99a22-49c8-4717-9f3b-170959979bfb None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:30.906577 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ffb99a22-49c8-4717-9f3b-170959979bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:31.019866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 677/1365] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:58:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:31.070307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9e388-aa71-444b-8b7d-444cf2e7f0c2 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:31.070632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9e388-aa71-444b-8b7d-444cf2e7f0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:31.070867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9e388-aa71-444b-8b7d-444cf2e7f0c2 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:31.072938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 689/1366] 10.222.0.44 () {54 vars in 961 bytes} [Tue Jul 12 21:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:31.078119 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9332cd6-e5d9-49ab-8dfd-e7df258651ba None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:31.078411 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9332cd6-e5d9-49ab-8dfd-e7df258651ba None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:31.078715 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9332cd6-e5d9-49ab-8dfd-e7df258651ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:31.159925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-a9332cd6-e5d9-49ab-8dfd-e7df258651ba None None] MFA Rules not processed for user `24915625354f418eb7bc4d5ada85fdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:31.161572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a9332cd6-e5d9-49ab-8dfd-e7df258651ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:31.201699 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:31.304277 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 678/1367] 10.222.0.44 () {58 vars in 1058 bytes} [Tue Jul 12 21:58:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:31.312580 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9e388-aa71-444b-8b7d-444cf2e7f0c2 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:31.314493 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2eb9e388-aa71-444b-8b7d-444cf2e7f0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:31.484467 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9e388-aa71-444b-8b7d-444cf2e7f0c2 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:31.507300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:31.616601 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d356c-49ef-49e6-b953-5d113c7a2bc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:31.617198 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d356c-49ef-49e6-b953-5d113c7a2bc4 service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:31.617488 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d356c-49ef-49e6-b953-5d113c7a2bc4 service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:31.617769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d356c-49ef-49e6-b953-5d113c7a2bc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:31.620009 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-188d356c-49ef-49e6-b953-5d113c7a2bc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:31.726245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d356c-49ef-49e6-b953-5d113c7a2bc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:31.728383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d356c-49ef-49e6-b953-5d113c7a2bc4 service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:31.749977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 690/1368] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:58:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:58:32.752598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d66df0-39a4-40a4-aabe-6654ba301844 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:32.753619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d66df0-39a4-40a4-aabe-6654ba301844 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:32.753926 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d66df0-39a4-40a4-aabe-6654ba301844 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:32.755500 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 679/1369] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:58:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:32.761005 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf06aea-67d4-45f7-aa7f-328a58d10b62 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:32.761468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf06aea-67d4-45f7-aa7f-328a58d10b62 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:32.761759 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf06aea-67d4-45f7-aa7f-328a58d10b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:32.794283 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:32.832942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-edf06aea-67d4-45f7-aa7f-328a58d10b62 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:32.834296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-edf06aea-67d4-45f7-aa7f-328a58d10b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:32.940558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 691/1370] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:58:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:33.285832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a6954-8512-4a8a-a28c-373d58525cb6 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:33.286124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a6954-8512-4a8a-a28c-373d58525cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:33.286401 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a6954-8512-4a8a-a28c-373d58525cb6 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:33.288001 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 680/1371] 10.222.0.44 () {54 vars in 963 bytes} [Tue Jul 12 21:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:33.293285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f39a4-a465-4adb-8c29-81530523a2e6 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:33.293652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f39a4-a465-4adb-8c29-81530523a2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:33.294023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f39a4-a465-4adb-8c29-81530523a2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:33.384388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-fd2f39a4-a465-4adb-8c29-81530523a2e6 None None] MFA Rules not processed for user `c53e1e801e5e40e794e4ccfff34ec988`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:33.385699 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-fd2f39a4-a465-4adb-8c29-81530523a2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:33.435365 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:33.470693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 692/1372] 10.222.0.44 () {58 vars in 1060 bytes} [Tue Jul 12 21:58:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:33.478329 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a6954-8512-4a8a-a28c-373d58525cb6 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:33.480008 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e90a6954-8512-4a8a-a28c-373d58525cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:33.597905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a6954-8512-4a8a-a28c-373d58525cb6 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:33.641204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:33.679805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8bb68-5288-4ed7-9e0a-babfb130dc47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:33.681378 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8bb68-5288-4ed7-9e0a-babfb130dc47 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:33.681666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8bb68-5288-4ed7-9e0a-babfb130dc47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:33.682073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8bb68-5288-4ed7-9e0a-babfb130dc47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:33.683791 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0eb8bb68-5288-4ed7-9e0a-babfb130dc47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:33.807975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8bb68-5288-4ed7-9e0a-babfb130dc47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:33.809489 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8bb68-5288-4ed7-9e0a-babfb130dc47 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:33.832109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 681/1373] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:58:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 21:58:33.918229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab550e-29ab-48ab-bc9e-b2d231b4030e None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:33.918697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab550e-29ab-48ab-bc9e-b2d231b4030e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:33.918790 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab550e-29ab-48ab-bc9e-b2d231b4030e None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:33.920154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 693/1374] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:33.926226 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc26ab4-1f4d-4c06-b3b4-a38c8784e227 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:33.926590 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc26ab4-1f4d-4c06-b3b4-a38c8784e227 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:33.927042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc26ab4-1f4d-4c06-b3b4-a38c8784e227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:34.009504 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-1bc26ab4-1f4d-4c06-b3b4-a38c8784e227 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:34.010775 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1bc26ab4-1f4d-4c06-b3b4-a38c8784e227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:34.032956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:34.140867 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 682/1375] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:34.174545 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab550e-29ab-48ab-bc9e-b2d231b4030e None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:34.176104 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-f6ab550e-29ab-48ab-bc9e-b2d231b4030e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:34.327865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab550e-29ab-48ab-bc9e-b2d231b4030e None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:34.379397 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:34.443734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e9f5b-051e-46c4-b6b2-24bd6e442bb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:34.444383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e9f5b-051e-46c4-b6b2-24bd6e442bb3 service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:34.444674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e9f5b-051e-46c4-b6b2-24bd6e442bb3 service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:34.444982 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e9f5b-051e-46c4-b6b2-24bd6e442bb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:34.447452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-be1e9f5b-051e-46c4-b6b2-24bd6e442bb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:34.539610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e9f5b-051e-46c4-b6b2-24bd6e442bb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:34.541789 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e9f5b-051e-46c4-b6b2-24bd6e442bb3 service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:34.562055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 694/1376] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:58:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:58:36.635783 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebad3d3-991b-49c9-bc63-5a11c4a69f5d None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:36.636333 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebad3d3-991b-49c9-bc63-5a11c4a69f5d None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:36.636333 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebad3d3-991b-49c9-bc63-5a11c4a69f5d None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:36.637743 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 683/1377] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:36.643098 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b573e7-5ca9-4f47-9346-ab60c18b06e7 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:36.643537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b573e7-5ca9-4f47-9346-ab60c18b06e7 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:36.643819 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b573e7-5ca9-4f47-9346-ab60c18b06e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:36.717141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-83b573e7-5ca9-4f47-9346-ab60c18b06e7 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:36.718598 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-83b573e7-5ca9-4f47-9346-ab60c18b06e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:36.725197 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:36.832127 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 695/1378] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:58:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:36.886689 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebad3d3-991b-49c9-bc63-5a11c4a69f5d None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:36.888477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-7ebad3d3-991b-49c9-bc63-5a11c4a69f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:37.063724 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebad3d3-991b-49c9-bc63-5a11c4a69f5d None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:37.067922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:37.188866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a6189-175e-42e5-9d94-316dcddaab9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:37.189769 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a6189-175e-42e5-9d94-316dcddaab9c service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:37.190052 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a6189-175e-42e5-9d94-316dcddaab9c service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:37.190329 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a6189-175e-42e5-9d94-316dcddaab9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:37.191991 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-193a6189-175e-42e5-9d94-316dcddaab9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:37.302689 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a6189-175e-42e5-9d94-316dcddaab9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:37.304318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a6189-175e-42e5-9d94-316dcddaab9c service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:37.326030 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 684/1379] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:58:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:58:39.461084 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627b28e-54a6-4ad2-93bd-4f4cd9ddc58c None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:39.461729 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627b28e-54a6-4ad2-93bd-4f4cd9ddc58c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:39.462018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627b28e-54a6-4ad2-93bd-4f4cd9ddc58c None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:39.463645 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 696/1380] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:39.469198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a58589-700c-4a71-bcd2-67c5c1a05906 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:39.469758 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a58589-700c-4a71-bcd2-67c5c1a05906 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:39.469758 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a58589-700c-4a71-bcd2-67c5c1a05906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:39.481822 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:39.553202 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-b1a58589-700c-4a71-bcd2-67c5c1a05906 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:39.554483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b1a58589-700c-4a71-bcd2-67c5c1a05906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:39.661397 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 685/1381] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:39.706102 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627b28e-54a6-4ad2-93bd-4f4cd9ddc58c None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:39.707625 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a627b28e-54a6-4ad2-93bd-4f4cd9ddc58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:39.872608 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:39.884953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627b28e-54a6-4ad2-93bd-4f4cd9ddc58c None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:40.020588 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93009068-3986-43b6-bfc7-e2a77976af73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:40.021264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93009068-3986-43b6-bfc7-e2a77976af73 service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:40.021610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93009068-3986-43b6-bfc7-e2a77976af73 service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:40.021900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93009068-3986-43b6-bfc7-e2a77976af73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:40.024503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-93009068-3986-43b6-bfc7-e2a77976af73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:40.130560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93009068-3986-43b6-bfc7-e2a77976af73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:40.132572 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93009068-3986-43b6-bfc7-e2a77976af73 service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:40.145296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:40.154720 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 697/1382] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:58:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:58:42.362951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb70dd2-ca49-49bf-8923-c8dec2a748d2 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:42.363497 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb70dd2-ca49-49bf-8923-c8dec2a748d2 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:42.363738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb70dd2-ca49-49bf-8923-c8dec2a748d2 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:42.365531 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 686/1383] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:58:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:42.372724 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2767d1-8e67-4457-987e-6e457e723175 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:42.373202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2767d1-8e67-4457-987e-6e457e723175 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:42.373299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2767d1-8e67-4457-987e-6e457e723175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:42.444049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-bd2767d1-8e67-4457-987e-6e457e723175 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:42.446479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-bd2767d1-8e67-4457-987e-6e457e723175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:42.572120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 698/1384] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:58:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:42.619319 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb70dd2-ca49-49bf-8923-c8dec2a748d2 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:42.620953 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6cb70dd2-ca49-49bf-8923-c8dec2a748d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:42.761309 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:42.828513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb70dd2-ca49-49bf-8923-c8dec2a748d2 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:42.955933 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca130a08-4971-4143-82c8-c7dc2ca938e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:42.956556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca130a08-4971-4143-82c8-c7dc2ca938e4 service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:42.956955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca130a08-4971-4143-82c8-c7dc2ca938e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:42.957257 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca130a08-4971-4143-82c8-c7dc2ca938e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:42.959050 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ca130a08-4971-4143-82c8-c7dc2ca938e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:43.013580 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:43.064239 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca130a08-4971-4143-82c8-c7dc2ca938e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:43.066967 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca130a08-4971-4143-82c8-c7dc2ca938e4 service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:43.089210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 687/1385] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:58:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:58:45.175585 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaf44a-cf59-405e-aca9-66fdd0fff467 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:45.176117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaf44a-cf59-405e-aca9-66fdd0fff467 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:45.176117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaf44a-cf59-405e-aca9-66fdd0fff467 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:45.177609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 699/1386] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:45.183353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007e593-5d3a-4e20-9148-816d3e738127 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:45.183855 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007e593-5d3a-4e20-9148-816d3e738127 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:45.183978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007e593-5d3a-4e20-9148-816d3e738127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:45.250928 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-6007e593-5d3a-4e20-9148-816d3e738127 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:45.252067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6007e593-5d3a-4e20-9148-816d3e738127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:45.355131 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 688/1387] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:58:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:45.406422 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcaf44a-cf59-405e-aca9-66fdd0fff467 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:45.407955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5dcaf44a-cf59-405e-aca9-66fdd0fff467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:45.502593 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:45.621800 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcaf44a-cf59-405e-aca9-66fdd0fff467 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:45.772500 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0761b-6c03-4de2-8f50-2175989a371e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:45.773190 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0761b-6c03-4de2-8f50-2175989a371e service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:45.773974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0761b-6c03-4de2-8f50-2175989a371e service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:45.774251 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0761b-6c03-4de2-8f50-2175989a371e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:45.775921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c9f0761b-6c03-4de2-8f50-2175989a371e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:45.815224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:45.878537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0761b-6c03-4de2-8f50-2175989a371e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:45.880047 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0761b-6c03-4de2-8f50-2175989a371e service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:45.902924 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 700/1388] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:58:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 21:58:48.040302 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a467eb-af24-4b01-9d18-0db8d299b79a None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:48.041076 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a467eb-af24-4b01-9d18-0db8d299b79a None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:48.041076 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a467eb-af24-4b01-9d18-0db8d299b79a None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:48.042266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 689/1389] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:58:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:48.047471 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056a43c-81cb-44da-9d74-78ed76abd839 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:48.047907 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056a43c-81cb-44da-9d74-78ed76abd839 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:48.048018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056a43c-81cb-44da-9d74-78ed76abd839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:48.114334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-5056a43c-81cb-44da-9d74-78ed76abd839 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:48.115540 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5056a43c-81cb-44da-9d74-78ed76abd839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:48.219366 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 701/1390] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:58:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:48.264854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a467eb-af24-4b01-9d18-0db8d299b79a None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:48.266505 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-20a467eb-af24-4b01-9d18-0db8d299b79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:48.298919 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:48.419082 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a467eb-af24-4b01-9d18-0db8d299b79a None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:48.527634 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:48.542147 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c976bb-9c96-4d3e-8a53-15c4fc3633b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:48.542802 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c976bb-9c96-4d3e-8a53-15c4fc3633b9 service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:48.543087 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c976bb-9c96-4d3e-8a53-15c4fc3633b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:48.543363 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c976bb-9c96-4d3e-8a53-15c4fc3633b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:48.545068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-34c976bb-9c96-4d3e-8a53-15c4fc3633b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:48.637463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c976bb-9c96-4d3e-8a53-15c4fc3633b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:48.639084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c976bb-9c96-4d3e-8a53-15c4fc3633b9 service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:48.657852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 690/1391] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:58:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:58:49.174362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeffee6-d5a2-4d75-b3fd-c75b2ae7a984 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:49.174935 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeffee6-d5a2-4d75-b3fd-c75b2ae7a984 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:49.174935 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeffee6-d5a2-4d75-b3fd-c75b2ae7a984 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:49.176644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 702/1392] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:49.182714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bc816-0b8f-4261-914c-0882150b4974 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:49.183215 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bc816-0b8f-4261-914c-0882150b4974 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:49.183313 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bc816-0b8f-4261-914c-0882150b4974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:49.251076 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-c84bc816-0b8f-4261-914c-0882150b4974 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:49.252197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c84bc816-0b8f-4261-914c-0882150b4974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:49.311142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:49.351917 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 691/1393] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:58:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:49.407981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b5891-73f6-4700-b37a-88562e7c0f99 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:49.408298 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b5891-73f6-4700-b37a-88562e7c0f99 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:49.408582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b5891-73f6-4700-b37a-88562e7c0f99 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:49.410076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 703/1394] 10.222.0.44 () {54 vars in 962 bytes} [Tue Jul 12 21:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:49.415525 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb0e80-bd4c-4fd0-9d7c-6318ee4dc554 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:49.415818 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb0e80-bd4c-4fd0-9d7c-6318ee4dc554 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:49.416099 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb0e80-bd4c-4fd0-9d7c-6318ee4dc554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:49.489711 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-e3cb0e80-bd4c-4fd0-9d7c-6318ee4dc554 None None] MFA Rules not processed for user `c5c9ed6f9fd542069cf914003b636c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:49.490915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e3cb0e80-bd4c-4fd0-9d7c-6318ee4dc554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:49.566746 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 692/1395] 10.222.0.44 () {58 vars in 1059 bytes} [Tue Jul 12 21:58:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:49.575190 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b5891-73f6-4700-b37a-88562e7c0f99 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:49.576793 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-103b5891-73f6-4700-b37a-88562e7c0f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:49.590547 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:49.685013 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b5891-73f6-4700-b37a-88562e7c0f99 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:49.760541 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c264118-7c8d-4ec0-92d0-1ee7daa18598 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5c9ed6f9fd542069cf914003b636c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:49.761225 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c264118-7c8d-4ec0-92d0-1ee7daa18598 service glance] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:49.761504 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c264118-7c8d-4ec0-92d0-1ee7daa18598 service glance] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:49.761779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c264118-7c8d-4ec0-92d0-1ee7daa18598 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:49.763428 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-7c264118-7c8d-4ec0-92d0-1ee7daa18598 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:49.836957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:49.881489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c264118-7c8d-4ec0-92d0-1ee7daa18598 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:49.882992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c264118-7c8d-4ec0-92d0-1ee7daa18598 service glance] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:49.901267 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 704/1396] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:58:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:58:50.635046 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1b073-c859-4723-84e3-4b2e751e87b2 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:50.635599 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1b073-c859-4723-84e3-4b2e751e87b2 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:50.635599 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1b073-c859-4723-84e3-4b2e751e87b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:50.703692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-6ab1b073-c859-4723-84e3-4b2e751e87b2 None None] MFA Rules not processed for user `8d6b823cc7dc434986f77739fa0259ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:50.704962 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6ab1b073-c859-4723-84e3-4b2e751e87b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:50.714882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:50.790861 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcadcf-e755-490d-bd75-f3c71b2bc3c4 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:50.791565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcadcf-e755-490d-bd75-f3c71b2bc3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:50.791565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcadcf-e755-490d-bd75-f3c71b2bc3c4 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:50.792958 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 705/1397] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:58:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:58:50.798130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 693/1398] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Jul 12 21:58:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:50.800898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a9b50-a78d-475c-b8a2-764d22465fd5 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:50.801190 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a9b50-a78d-475c-b8a2-764d22465fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:50.801467 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a9b50-a78d-475c-b8a2-764d22465fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:50.841803 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71e71e-0000-4e3b-a06d-53c73ad59da9 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:50.842094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71e71e-0000-4e3b-a06d-53c73ad59da9 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:50.842374 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71e71e-0000-4e3b-a06d-53c73ad59da9 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:50.843746 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 694/1399] 10.222.0.44 () {54 vars in 942 bytes} [Tue Jul 12 21:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:58:50.850036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5526612-5e76-4482-87ed-113c138bf0e9 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:50.850328 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5526612-5e76-4482-87ed-113c138bf0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:50.850617 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5526612-5e76-4482-87ed-113c138bf0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:50.881976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-eb0a9b50-a78d-475c-b8a2-764d22465fd5 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:50.884099 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-eb0a9b50-a78d-475c-b8a2-764d22465fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:50.934232 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-d5526612-5e76-4482-87ed-113c138bf0e9 None None] MFA Rules not processed for user `837bb5ac18554dfdbbcc776689639494`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:50.935461 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d5526612-5e76-4482-87ed-113c138bf0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:51.007121 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 706/1400] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:58:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:51.032305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:51.053004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a9b50-a78d-475c-b8a2-764d22465fd5 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:51.054228 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-eb0a9b50-a78d-475c-b8a2-764d22465fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:51.056594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 695/1401] 10.222.0.44 () {58 vars in 1039 bytes} [Tue Jul 12 21:58:50 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 21:58:51.066161 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5526612-5e76-4482-87ed-113c138bf0e9 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:51.075793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5526612-5e76-4482-87ed-113c138bf0e9 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:51.082349 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:51.174978 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a9b50-a78d-475c-b8a2-764d22465fd5 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:51.180348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36617d0d-5c10-4ce0-a1a9-f6ead3ba3315 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837bb5ac18554dfdbbcc776689639494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:51.181026 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36617d0d-5c10-4ce0-a1a9-f6ead3ba3315 service swift] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:51.181312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36617d0d-5c10-4ce0-a1a9-f6ead3ba3315 service swift] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:51.181581 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36617d0d-5c10-4ce0-a1a9-f6ead3ba3315 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:51.183286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-36617d0d-5c10-4ce0-a1a9-f6ead3ba3315 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:51.272777 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36617d0d-5c10-4ce0-a1a9-f6ead3ba3315 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:51.274626 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36617d0d-5c10-4ce0-a1a9-f6ead3ba3315 service swift] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:51.284202 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 696/1402] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Jul 12 21:58:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 21:58:51.296047 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e561d1-a347-4ab8-b4a0-85ee944a83d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:51.296731 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e561d1-a347-4ab8-b4a0-85ee944a83d4 service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:51.297141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e561d1-a347-4ab8-b4a0-85ee944a83d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:51.297421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e561d1-a347-4ab8-b4a0-85ee944a83d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:51.299300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-46e561d1-a347-4ab8-b4a0-85ee944a83d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:51.320042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:51.399831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e561d1-a347-4ab8-b4a0-85ee944a83d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:51.401588 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e561d1-a347-4ab8-b4a0-85ee944a83d4 service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:51.425486 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 707/1403] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:58:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 21:58:53.532555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a765e5d-c0af-44b1-8a49-2f17fd715ac4 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:53.533168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a765e5d-c0af-44b1-8a49-2f17fd715ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:53.533168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a765e5d-c0af-44b1-8a49-2f17fd715ac4 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:53.534522 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 697/1404] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:53.539867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c9f74-6257-49b5-84cf-1ef07853eb96 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:53.540355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c9f74-6257-49b5-84cf-1ef07853eb96 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:53.540355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c9f74-6257-49b5-84cf-1ef07853eb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:53.613462 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-a53c9f74-6257-49b5-84cf-1ef07853eb96 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:53.616054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a53c9f74-6257-49b5-84cf-1ef07853eb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:53.724957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:53.733177 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 708/1405] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:53.780478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a765e5d-c0af-44b1-8a49-2f17fd715ac4 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:53.782078 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1a765e5d-c0af-44b1-8a49-2f17fd715ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:53.796255 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:53.956317 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a765e5d-c0af-44b1-8a49-2f17fd715ac4 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:54.064635 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:54.101297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f86a0d-b361-46d0-9ce7-bbff1aed0527 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:54.102350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f86a0d-b361-46d0-9ce7-bbff1aed0527 service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:54.102632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f86a0d-b361-46d0-9ce7-bbff1aed0527 service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:54.102904 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f86a0d-b361-46d0-9ce7-bbff1aed0527 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:54.104558 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-32f86a0d-b361-46d0-9ce7-bbff1aed0527 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:54.206120 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f86a0d-b361-46d0-9ce7-bbff1aed0527 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:54.207720 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f86a0d-b361-46d0-9ce7-bbff1aed0527 service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:54.227793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 698/1406] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:58:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:58:56.384972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158c6e4-4537-48d6-a998-e6fdb210be18 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:56.385614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158c6e4-4537-48d6-a998-e6fdb210be18 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:56.385614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158c6e4-4537-48d6-a998-e6fdb210be18 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:56.386928 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 709/1407] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 21:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:58:56.392210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bdde5-3cff-47b6-a7eb-786bfed797b5 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:56.392641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bdde5-3cff-47b6-a7eb-786bfed797b5 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:56.392845 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bdde5-3cff-47b6-a7eb-786bfed797b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:56.467783 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-069bdde5-3cff-47b6-a7eb-786bfed797b5 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:58:56.469075 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-069bdde5-3cff-47b6-a7eb-786bfed797b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:56.526465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:56.592814 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 699/1408] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 21:58:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:58:56.630784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158c6e4-4537-48d6-a998-e6fdb210be18 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:58:56.633640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-3158c6e4-4537-48d6-a998-e6fdb210be18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:56.802885 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158c6e4-4537-48d6-a998-e6fdb210be18 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:58:56.877188 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:58:56.937997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebe61c-4858-41fe-9149-05df97bb9175 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:58:56.938655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebe61c-4858-41fe-9149-05df97bb9175 service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:58:56.939177 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebe61c-4858-41fe-9149-05df97bb9175 service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:58:56.939458 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebe61c-4858-41fe-9149-05df97bb9175 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:58:56.941671 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c4ebe61c-4858-41fe-9149-05df97bb9175 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:58:57.053841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebe61c-4858-41fe-9149-05df97bb9175 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:58:57.056763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebe61c-4858-41fe-9149-05df97bb9175 service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:58:57.079371 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 710/1409] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:58:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:06.584666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=23 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:07.076919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=64 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:14.088601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c28e9-b9ba-4487-90e5-4d887f96e67e None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:14.089219 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c28e9-b9ba-4487-90e5-4d887f96e67e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:14.089219 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c28e9-b9ba-4487-90e5-4d887f96e67e None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:14.090572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 700/1410] 10.222.0.44 () {54 vars in 918 bytes} [Tue Jul 12 21:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 21:59:14.095911 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c06d00-bfa8-48d5-b48e-a546424699a3 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:14.096326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c06d00-bfa8-48d5-b48e-a546424699a3 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:14.096445 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c06d00-bfa8-48d5-b48e-a546424699a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:14.186483 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-c5c06d00-bfa8-48d5-b48e-a546424699a3 None None] MFA Rules not processed for user `a1f1ef52f8eb42a59a50a69873b9ffaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:14.188043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c5c06d00-bfa8-48d5-b48e-a546424699a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:14.295694 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 711/1411] 10.222.0.44 () {58 vars in 1015 bytes} [Tue Jul 12 21:59:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 21:59:14.310136 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fa500-1317-467f-96ba-750e06f7bc69 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:14.310431 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fa500-1317-467f-96ba-750e06f7bc69 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:14.310734 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fa500-1317-467f-96ba-750e06f7bc69 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:14.312108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 701/1412] 10.222.0.44 () {54 vars in 918 bytes} [Tue Jul 12 21:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:14.316903 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977e257-c111-4884-8743-c41c8ca536a9 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:14.317196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977e257-c111-4884-8743-c41c8ca536a9 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:14.317479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977e257-c111-4884-8743-c41c8ca536a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:14.387738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:14.390766 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-d977e257-c111-4884-8743-c41c8ca536a9 None None] MFA Rules not processed for user `24915625354f418eb7bc4d5ada85fdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:14.392638 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d977e257-c111-4884-8743-c41c8ca536a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:14.510975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 712/1413] 10.222.0.44 () {58 vars in 1015 bytes} [Tue Jul 12 21:59:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:17.050982 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a96274-c817-4ed3-ad07-f5c5d7d714b9 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:17.051563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a96274-c817-4ed3-ad07-f5c5d7d714b9 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:17.051563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a96274-c817-4ed3-ad07-f5c5d7d714b9 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:17.052953 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 21:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:17.058729 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc41991-cb6a-4109-b845-51b907303788 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:17.059127 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc41991-cb6a-4109-b845-51b907303788 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:17.059321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc41991-cb6a-4109-b845-51b907303788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:17.123524 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-1bc41991-cb6a-4109-b845-51b907303788 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:17.124750 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-1bc41991-cb6a-4109-b845-51b907303788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:17.180950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:17.224845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 21:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:17.268328 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a96274-c817-4ed3-ad07-f5c5d7d714b9 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:17.270095 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-16a96274-c817-4ed3-ad07-f5c5d7d714b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:17.414142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a96274-c817-4ed3-ad07-f5c5d7d714b9 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:17.485963 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:17.526216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83649fbd-6483-4958-9d87-1202317c7116 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:17.526878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83649fbd-6483-4958-9d87-1202317c7116 service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:17.527163 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83649fbd-6483-4958-9d87-1202317c7116 service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:17.527440 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83649fbd-6483-4958-9d87-1202317c7116 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:17.529344 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-83649fbd-6483-4958-9d87-1202317c7116 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:17.621215 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83649fbd-6483-4958-9d87-1202317c7116 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:17.622981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83649fbd-6483-4958-9d87-1202317c7116 service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:17.640580 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 703/1416] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:18.543527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6c366-2e3a-438d-8f35-a05e18228761 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:18.544094 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6c366-2e3a-438d-8f35-a05e18228761 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:18.544094 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6c366-2e3a-438d-8f35-a05e18228761 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:18.545472 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 21:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:18.550474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4efcb-4303-498f-8089-50124694835c None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:18.550916 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4efcb-4303-498f-8089-50124694835c None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:18.551022 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4efcb-4303-498f-8089-50124694835c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:18.625241 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-76f4efcb-4303-498f-8089-50124694835c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:18.626463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-76f4efcb-4303-498f-8089-50124694835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:18.653069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:18.727465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 21:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:18.878410 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6c366-2e3a-438d-8f35-a05e18228761 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:18.879957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ace6c366-2e3a-438d-8f35-a05e18228761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:19.029465 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6c366-2e3a-438d-8f35-a05e18228761 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:19.072577 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:19.143679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd4c51-d4a7-4cb9-ab02-06377a1e3a6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:19.144329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd4c51-d4a7-4cb9-ab02-06377a1e3a6b service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:19.144613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd4c51-d4a7-4cb9-ab02-06377a1e3a6b service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:19.144915 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd4c51-d4a7-4cb9-ab02-06377a1e3a6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:19.146564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-32cd4c51-d4a7-4cb9-ab02-06377a1e3a6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:19.239970 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd4c51-d4a7-4cb9-ab02-06377a1e3a6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:19.241638 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd4c51-d4a7-4cb9-ab02-06377a1e3a6b service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:19.259667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 715/1419] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:20.142176 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a233aa9-0c59-412e-aadb-9a06fb15f6c9 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:20.142788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a233aa9-0c59-412e-aadb-9a06fb15f6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:20.142788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a233aa9-0c59-412e-aadb-9a06fb15f6c9 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:20.144158 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 21:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:20.149215 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5338c-0bee-4115-9522-314d15fe8a90 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:20.149637 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5338c-0bee-4115-9522-314d15fe8a90 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:20.149769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5338c-0bee-4115-9522-314d15fe8a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:20.203015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:20.225142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-3bd5338c-0bee-4115-9522-314d15fe8a90 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:20.226421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-3bd5338c-0bee-4115-9522-314d15fe8a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:20.325538 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 21:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:20.479955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a233aa9-0c59-412e-aadb-9a06fb15f6c9 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:20.481466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4a233aa9-0c59-412e-aadb-9a06fb15f6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:20.627004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a233aa9-0c59-412e-aadb-9a06fb15f6c9 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:20.639239 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:20.740624 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10428c3a-8a87-4b3a-9b3a-a964f59a53e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:20.741249 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10428c3a-8a87-4b3a-9b3a-a964f59a53e6 service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:20.741532 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10428c3a-8a87-4b3a-9b3a-a964f59a53e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:20.741815 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10428c3a-8a87-4b3a-9b3a-a964f59a53e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:20.743485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-10428c3a-8a87-4b3a-9b3a-a964f59a53e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:20.832631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10428c3a-8a87-4b3a-9b3a-a964f59a53e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:20.834292 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10428c3a-8a87-4b3a-9b3a-a964f59a53e6 service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:20.852266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 706/1422] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:21.722252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f20c6-350e-4e9e-9cd2-ffac65504385 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:21.722830 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f20c6-350e-4e9e-9cd2-ffac65504385 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:21.722830 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f20c6-350e-4e9e-9cd2-ffac65504385 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:21.724169 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 21:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:21.729941 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90335ac-b551-4ef5-a30a-95904b61b38e None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:21.730339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90335ac-b551-4ef5-a30a-95904b61b38e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:21.730517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90335ac-b551-4ef5-a30a-95904b61b38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:21.750033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:21.806694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-d90335ac-b551-4ef5-a30a-95904b61b38e None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:21.807917 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d90335ac-b551-4ef5-a30a-95904b61b38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:21.907306 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 21:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:22.055958 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f20c6-350e-4e9e-9cd2-ffac65504385 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:22.057502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-db7f20c6-350e-4e9e-9cd2-ffac65504385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:22.170286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:22.207742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f20c6-350e-4e9e-9cd2-ffac65504385 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:22.332943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728aa7f-3394-4979-bfd2-48aa24416f13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:22.333498 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728aa7f-3394-4979-bfd2-48aa24416f13 service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:22.333783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728aa7f-3394-4979-bfd2-48aa24416f13 service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:22.334060 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728aa7f-3394-4979-bfd2-48aa24416f13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:22.335746 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e728aa7f-3394-4979-bfd2-48aa24416f13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:22.411290 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:22.425785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728aa7f-3394-4979-bfd2-48aa24416f13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:22.427398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728aa7f-3394-4979-bfd2-48aa24416f13 service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:22.447259 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 718/1425] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:23.323170 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253dc60-0247-40f2-b718-babf2b56cfad None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:23.323780 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253dc60-0247-40f2-b718-babf2b56cfad None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:23.323780 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253dc60-0247-40f2-b718-babf2b56cfad None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:23.325288 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 21:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:23.330237 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101f813-8fea-4d4f-91cc-b0d799eeac43 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:23.330674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101f813-8fea-4d4f-91cc-b0d799eeac43 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:23.330783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101f813-8fea-4d4f-91cc-b0d799eeac43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:23.404512 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-6101f813-8fea-4d4f-91cc-b0d799eeac43 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:23.405764 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-6101f813-8fea-4d4f-91cc-b0d799eeac43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:23.504619 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 21:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:23.655795 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253dc60-0247-40f2-b718-babf2b56cfad None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:23.657256 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-f253dc60-0247-40f2-b718-babf2b56cfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:23.742706 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:23.804606 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253dc60-0247-40f2-b718-babf2b56cfad None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:23.921840 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f879b4-de1f-4cff-8832-39a3f7605c9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:23.922403 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f879b4-de1f-4cff-8832-39a3f7605c9c service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:23.922690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f879b4-de1f-4cff-8832-39a3f7605c9c service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:23.922968 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f879b4-de1f-4cff-8832-39a3f7605c9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:23.924632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e7f879b4-de1f-4cff-8832-39a3f7605c9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:23.973820 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:24.016087 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f879b4-de1f-4cff-8832-39a3f7605c9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:24.017727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f879b4-de1f-4cff-8832-39a3f7605c9c service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:24.036145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 709/1428] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:24.839808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0203e87-a1b3-401b-8d73-3cbce071d557 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:24.840439 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0203e87-a1b3-401b-8d73-3cbce071d557 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:24.840539 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0203e87-a1b3-401b-8d73-3cbce071d557 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:24.841948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 21:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:24.846870 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c06bc-9cfa-4c4a-986d-20ea0583a871 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:24.847355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c06bc-9cfa-4c4a-986d-20ea0583a871 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:24.847460 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c06bc-9cfa-4c4a-986d-20ea0583a871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:24.922882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-4e5c06bc-9cfa-4c4a-986d-20ea0583a871 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:24.924107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4e5c06bc-9cfa-4c4a-986d-20ea0583a871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:25.024734 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 21:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:25.175159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0203e87-a1b3-401b-8d73-3cbce071d557 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:25.176611 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d0203e87-a1b3-401b-8d73-3cbce071d557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:25.228606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:25.327142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0203e87-a1b3-401b-8d73-3cbce071d557 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:25.439033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950de72-59bd-4e6d-95d4-022b59bf0c75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:25.439839 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950de72-59bd-4e6d-95d4-022b59bf0c75 service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:25.440116 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950de72-59bd-4e6d-95d4-022b59bf0c75 service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:25.440387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950de72-59bd-4e6d-95d4-022b59bf0c75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:25.442059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e950de72-59bd-4e6d-95d4-022b59bf0c75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:25.449783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:25.532080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950de72-59bd-4e6d-95d4-022b59bf0c75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:25.533728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950de72-59bd-4e6d-95d4-022b59bf0c75 service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:25.551880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 721/1431] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:26.369456 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11ee61-6b90-452a-8029-b51511d2ddef None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:26.370090 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11ee61-6b90-452a-8029-b51511d2ddef None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:26.370090 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11ee61-6b90-452a-8029-b51511d2ddef None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:26.371389 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 21:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:26.376259 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b554a-f87a-4eab-9e6c-70f69476e4aa None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:26.376745 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b554a-f87a-4eab-9e6c-70f69476e4aa None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:26.376850 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b554a-f87a-4eab-9e6c-70f69476e4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:26.450308 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-f24b554a-f87a-4eab-9e6c-70f69476e4aa None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:26.451506 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-f24b554a-f87a-4eab-9e6c-70f69476e4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:26.519172 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:26.549303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 21:59:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:26.702657 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11ee61-6b90-452a-8029-b51511d2ddef None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:26.704102 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8e11ee61-6b90-452a-8029-b51511d2ddef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:26.723231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:26.854991 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11ee61-6b90-452a-8029-b51511d2ddef None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:26.943382 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:26.972254 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59722f-1740-4143-a005-9c97b8b621b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:26.972940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59722f-1740-4143-a005-9c97b8b621b3 service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:26.973225 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59722f-1740-4143-a005-9c97b8b621b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:26.973696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59722f-1740-4143-a005-9c97b8b621b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:26.975363 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-5d59722f-1740-4143-a005-9c97b8b621b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:27.067690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59722f-1740-4143-a005-9c97b8b621b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:27.069353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59722f-1740-4143-a005-9c97b8b621b3 service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:27.087848 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 712/1434] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:27.928659 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaed3a-6149-4164-9ebf-00cdf7f2f4fb None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:27.929268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaed3a-6149-4164-9ebf-00cdf7f2f4fb None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:27.929268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaed3a-6149-4164-9ebf-00cdf7f2f4fb None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:27.930531 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 21:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:27.935543 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee14b4-589e-46c4-9988-787de97b639c None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:27.935978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee14b4-589e-46c4-9988-787de97b639c None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:27.936088 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee14b4-589e-46c4-9988-787de97b639c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:28.012668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-e2ee14b4-589e-46c4-9988-787de97b639c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:28.013918 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e2ee14b4-589e-46c4-9988-787de97b639c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:28.054757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:28.114578 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 21:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:28.266703 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaaed3a-6149-4164-9ebf-00cdf7f2f4fb None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:28.268222 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-efaaed3a-6149-4164-9ebf-00cdf7f2f4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:28.415953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaaed3a-6149-4164-9ebf-00cdf7f2f4fb None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:28.469349 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:28.529163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eae03e-2b68-4c03-9d98-46be9abd4419 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:28.529730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eae03e-2b68-4c03-9d98-46be9abd4419 service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:28.530020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eae03e-2b68-4c03-9d98-46be9abd4419 service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:28.530327 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eae03e-2b68-4c03-9d98-46be9abd4419 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:28.531983 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-51eae03e-2b68-4c03-9d98-46be9abd4419 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:28.621183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eae03e-2b68-4c03-9d98-46be9abd4419 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:28.622773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eae03e-2b68-4c03-9d98-46be9abd4419 service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:28.641143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 724/1437] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:29.486790 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa91cb7-ad1d-4fc3-a22f-2a5d869c5f74 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:29.487409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa91cb7-ad1d-4fc3-a22f-2a5d869c5f74 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:29.487409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa91cb7-ad1d-4fc3-a22f-2a5d869c5f74 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:29.488760 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 21:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:29.495496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb19349-3635-4c3b-973e-0d3c1d39edc5 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:29.495948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb19349-3635-4c3b-973e-0d3c1d39edc5 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:29.496053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb19349-3635-4c3b-973e-0d3c1d39edc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:29.571834 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-9eb19349-3635-4c3b-973e-0d3c1d39edc5 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:29.573077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-9eb19349-3635-4c3b-973e-0d3c1d39edc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:29.575097 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:29.687126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 21:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:29.836160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa91cb7-ad1d-4fc3-a22f-2a5d869c5f74 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:29.837712 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-baa91cb7-ad1d-4fc3-a22f-2a5d869c5f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:29.989031 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa91cb7-ad1d-4fc3-a22f-2a5d869c5f74 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:30.017362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:30.103838 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de3f13-2d11-4e15-a217-5ae056ee20b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:30.104488 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de3f13-2d11-4e15-a217-5ae056ee20b4 service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:30.104815 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de3f13-2d11-4e15-a217-5ae056ee20b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:30.105097 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de3f13-2d11-4e15-a217-5ae056ee20b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:30.106757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e6de3f13-2d11-4e15-a217-5ae056ee20b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:30.196232 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de3f13-2d11-4e15-a217-5ae056ee20b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:30.197876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de3f13-2d11-4e15-a217-5ae056ee20b4 service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:30.215844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 715/1440] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:32.169604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893126ad-de49-4cfc-83fa-4aceeb4358ce None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:32.170163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893126ad-de49-4cfc-83fa-4aceeb4358ce None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:32.170163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893126ad-de49-4cfc-83fa-4aceeb4358ce None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:32.171526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 21:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:32.176745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93997c-aeba-4d5f-b96b-7c8b0db3dc17 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:32.177196 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93997c-aeba-4d5f-b96b-7c8b0db3dc17 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:32.177303 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93997c-aeba-4d5f-b96b-7c8b0db3dc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:32.219461 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:32.245396 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-cc93997c-aeba-4d5f-b96b-7c8b0db3dc17 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:32.246619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-cc93997c-aeba-4d5f-b96b-7c8b0db3dc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:32.356246 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 21:59:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:32.394858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893126ad-de49-4cfc-83fa-4aceeb4358ce None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:32.396399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-893126ad-de49-4cfc-83fa-4aceeb4358ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:32.502190 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893126ad-de49-4cfc-83fa-4aceeb4358ce None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:32.533966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:32.577742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081ebc9-fae6-46c2-8d2c-1e9c67901dde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:32.578389 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081ebc9-fae6-46c2-8d2c-1e9c67901dde service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:32.578679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081ebc9-fae6-46c2-8d2c-1e9c67901dde service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:32.578959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081ebc9-fae6-46c2-8d2c-1e9c67901dde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:32.580622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c081ebc9-fae6-46c2-8d2c-1e9c67901dde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:32.689142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081ebc9-fae6-46c2-8d2c-1e9c67901dde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:32.690830 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081ebc9-fae6-46c2-8d2c-1e9c67901dde service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:32.709117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 727/1443] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:34.894364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c156d-c424-4c59-8913-6c1775201472 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:34.894902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c156d-c424-4c59-8913-6c1775201472 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:34.895001 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c156d-c424-4c59-8913-6c1775201472 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:34.896304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 21:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:34.902029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42554e90-1d8c-4a59-b51d-e90f18daafde None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:34.902448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42554e90-1d8c-4a59-b51d-e90f18daafde None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:34.902595 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42554e90-1d8c-4a59-b51d-e90f18daafde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:34.967325 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-42554e90-1d8c-4a59-b51d-e90f18daafde None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:34.968561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-42554e90-1d8c-4a59-b51d-e90f18daafde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:34.981992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:35.067939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 21:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:35.105350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c156d-c424-4c59-8913-6c1775201472 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:35.106976 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-3a4c156d-c424-4c59-8913-6c1775201472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:35.212373 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c156d-c424-4c59-8913-6c1775201472 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:35.224480 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:35.286269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da8237-e8d4-4853-b524-07d0fcf00c9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:35.286917 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da8237-e8d4-4853-b524-07d0fcf00c9b service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:35.287211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da8237-e8d4-4853-b524-07d0fcf00c9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:35.287491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da8237-e8d4-4853-b524-07d0fcf00c9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:35.289799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-63da8237-e8d4-4853-b524-07d0fcf00c9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:35.396965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da8237-e8d4-4853-b524-07d0fcf00c9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:35.399070 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da8237-e8d4-4853-b524-07d0fcf00c9b service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:35.417475 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 718/1446] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:37.887796 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92c9d9-00ab-40be-b2db-eb83bcc59d69 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:37.888383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92c9d9-00ab-40be-b2db-eb83bcc59d69 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:37.888383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92c9d9-00ab-40be-b2db-eb83bcc59d69 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:37.889783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 21:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:37.894983 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d67b6-b53b-43fe-96a3-9fb6a5d9913b None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:37.895407 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d67b6-b53b-43fe-96a3-9fb6a5d9913b None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:37.895551 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d67b6-b53b-43fe-96a3-9fb6a5d9913b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:37.942211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:37.962167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-cc0d67b6-b53b-43fe-96a3-9fb6a5d9913b None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:37.963397 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-cc0d67b6-b53b-43fe-96a3-9fb6a5d9913b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:38.082377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 21:59:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:38.120853 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92c9d9-00ab-40be-b2db-eb83bcc59d69 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:38.122417 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-fe92c9d9-00ab-40be-b2db-eb83bcc59d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:38.228958 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92c9d9-00ab-40be-b2db-eb83bcc59d69 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:38.269875 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:38.303148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51425e-69ec-44dc-848d-185f37189ec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:38.304343 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51425e-69ec-44dc-848d-185f37189ec8 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:38.304630 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51425e-69ec-44dc-848d-185f37189ec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:38.304929 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51425e-69ec-44dc-848d-185f37189ec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:38.306583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-6d51425e-69ec-44dc-848d-185f37189ec8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:38.416393 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51425e-69ec-44dc-848d-185f37189ec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:38.417853 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51425e-69ec-44dc-848d-185f37189ec8 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:38.435895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 730/1449] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:40.602550 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879a4dc-0026-4e50-82be-9f947b89e923 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:40.603107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879a4dc-0026-4e50-82be-9f947b89e923 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:40.603107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879a4dc-0026-4e50-82be-9f947b89e923 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:40.604609 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 21:59:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:40.609948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b96e5-eb57-4610-9e0d-aea3dd84b2d7 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:40.610396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b96e5-eb57-4610-9e0d-aea3dd84b2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:40.610493 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b96e5-eb57-4610-9e0d-aea3dd84b2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:40.675710 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-264b96e5-eb57-4610-9e0d-aea3dd84b2d7 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:40.676985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-264b96e5-eb57-4610-9e0d-aea3dd84b2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:40.698845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:40.777628 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 21:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:40.816091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879a4dc-0026-4e50-82be-9f947b89e923 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:40.817705 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d879a4dc-0026-4e50-82be-9f947b89e923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:40.923742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879a4dc-0026-4e50-82be-9f947b89e923 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:40.943427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:41.116768 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33778f-776a-4ea3-b198-d7e616856a34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:41.117413 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33778f-776a-4ea3-b198-d7e616856a34 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:41.117698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33778f-776a-4ea3-b198-d7e616856a34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:41.117977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33778f-776a-4ea3-b198-d7e616856a34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:41.119678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-fc33778f-776a-4ea3-b198-d7e616856a34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:41.232238 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33778f-776a-4ea3-b198-d7e616856a34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:41.233635 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33778f-776a-4ea3-b198-d7e616856a34 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:41.252379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 721/1452] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:43.533222 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abf668-0cdf-49a3-837e-73a39c0dba68 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:43.533799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abf668-0cdf-49a3-837e-73a39c0dba68 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:43.533799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abf668-0cdf-49a3-837e-73a39c0dba68 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:43.535155 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 21:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:43.540650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228e49e-7f7d-4400-8087-b214d792966c None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:43.541121 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228e49e-7f7d-4400-8087-b214d792966c None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:43.541229 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228e49e-7f7d-4400-8087-b214d792966c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:43.603863 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:43.607271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-0228e49e-7f7d-4400-8087-b214d792966c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:43.608939 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0228e49e-7f7d-4400-8087-b214d792966c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:43.705612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 21:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:43.744587 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0abf668-0cdf-49a3-837e-73a39c0dba68 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:43.746172 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b0abf668-0cdf-49a3-837e-73a39c0dba68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:43.850134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0abf668-0cdf-49a3-837e-73a39c0dba68 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:43.902755 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:43.927986 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bafb53-ccd9-46ef-87d6-594cbfc9ec4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:43.929242 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bafb53-ccd9-46ef-87d6-594cbfc9ec4c service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:43.929535 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bafb53-ccd9-46ef-87d6-594cbfc9ec4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:43.929976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bafb53-ccd9-46ef-87d6-594cbfc9ec4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:43.931652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-53bafb53-ccd9-46ef-87d6-594cbfc9ec4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:44.043898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bafb53-ccd9-46ef-87d6-594cbfc9ec4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:44.045380 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bafb53-ccd9-46ef-87d6-594cbfc9ec4c service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:44.064502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 733/1455] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:46.200415 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6926e92-7129-4b34-b5c8-8f52772b6f1d None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:46.200992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6926e92-7129-4b34-b5c8-8f52772b6f1d None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:46.200992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6926e92-7129-4b34-b5c8-8f52772b6f1d None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:46.202306 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 21:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:46.207771 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07a09f-4767-40cf-ac64-07e235415bb2 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:46.208196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07a09f-4767-40cf-ac64-07e235415bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:46.208338 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07a09f-4767-40cf-ac64-07e235415bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:46.274249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-1e07a09f-4767-40cf-ac64-07e235415bb2 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:46.275466 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-1e07a09f-4767-40cf-ac64-07e235415bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:46.306028 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:46.376004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 21:59:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:46.415019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6926e92-7129-4b34-b5c8-8f52772b6f1d None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:46.416610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c6926e92-7129-4b34-b5c8-8f52772b6f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:46.521934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6926e92-7129-4b34-b5c8-8f52772b6f1d None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:46.555475 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:46.602234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c0360-aaf0-4a38-bfcc-aaf8ca1d5cfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:46.602885 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c0360-aaf0-4a38-bfcc-aaf8ca1d5cfc service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:46.603176 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c0360-aaf0-4a38-bfcc-aaf8ca1d5cfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:46.603455 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c0360-aaf0-4a38-bfcc-aaf8ca1d5cfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:46.605155 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d63c0360-aaf0-4a38-bfcc-aaf8ca1d5cfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:46.716943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c0360-aaf0-4a38-bfcc-aaf8ca1d5cfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:46.718397 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c0360-aaf0-4a38-bfcc-aaf8ca1d5cfc service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:46.738261 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 724/1458] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:48.837915 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483e7bd-8b6b-4bd8-a31e-67c5a3f9291f None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:48.838546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483e7bd-8b6b-4bd8-a31e-67c5a3f9291f None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:48.838546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483e7bd-8b6b-4bd8-a31e-67c5a3f9291f None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:48.839874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 21:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:48.845359 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd2b39-f66a-4017-a740-742fa99b1a56 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:48.845767 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd2b39-f66a-4017-a740-742fa99b1a56 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:48.845913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd2b39-f66a-4017-a740-742fa99b1a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:48.910678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-53cd2b39-f66a-4017-a740-742fa99b1a56 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:48.911889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-53cd2b39-f66a-4017-a740-742fa99b1a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:48.920184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:49.010174 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 21:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:49.047342 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483e7bd-8b6b-4bd8-a31e-67c5a3f9291f None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:49.048964 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-7483e7bd-8b6b-4bd8-a31e-67c5a3f9291f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:49.155122 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483e7bd-8b6b-4bd8-a31e-67c5a3f9291f None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:49.212445 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:49.231492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62016e8-504a-4c6d-9094-bfe37368917d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:49.232140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62016e8-504a-4c6d-9094-bfe37368917d service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:49.232426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62016e8-504a-4c6d-9094-bfe37368917d service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:49.232729 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62016e8-504a-4c6d-9094-bfe37368917d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:49.234395 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b62016e8-504a-4c6d-9094-bfe37368917d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:49.344279 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62016e8-504a-4c6d-9094-bfe37368917d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:49.345709 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62016e8-504a-4c6d-9094-bfe37368917d service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:49.364417 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 736/1461] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:51.480051 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d7223-bd8d-4255-95a9-4ffc2c26f1bc None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:51.480634 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d7223-bd8d-4255-95a9-4ffc2c26f1bc None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:51.480634 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d7223-bd8d-4255-95a9-4ffc2c26f1bc None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:51.482008 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 21:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:51.487226 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd0df3-0111-4903-9bc5-15b7e7391849 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:51.487661 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd0df3-0111-4903-9bc5-15b7e7391849 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:51.487792 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd0df3-0111-4903-9bc5-15b7e7391849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:51.552494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-e1bd0df3-0111-4903-9bc5-15b7e7391849 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:51.554387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e1bd0df3-0111-4903-9bc5-15b7e7391849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:51.593170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:51.651514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 21:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 21:59:51.690014 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d7223-bd8d-4255-95a9-4ffc2c26f1bc None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:51.691812 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-2d1d7223-bd8d-4255-95a9-4ffc2c26f1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:51.797054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d7223-bd8d-4255-95a9-4ffc2c26f1bc None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:51.836204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:51.873970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bdf686-d811-4753-9aee-7c2608490131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:51.874657 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdf686-d811-4753-9aee-7c2608490131 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:51.874945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdf686-d811-4753-9aee-7c2608490131 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:51.875223 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdf686-d811-4753-9aee-7c2608490131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:51.876958 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-76bdf686-d811-4753-9aee-7c2608490131 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:51.987017 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bdf686-d811-4753-9aee-7c2608490131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:51.988469 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bdf686-d811-4753-9aee-7c2608490131 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:52.008199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 727/1464] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:54.092698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d1093-e9b1-424b-94c0-664f340a4b1e None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:54.093249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d1093-e9b1-424b-94c0-664f340a4b1e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:54.093249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d1093-e9b1-424b-94c0-664f340a4b1e None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:54.094648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 21:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:54.099831 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136a382-deb0-4bd4-9cb3-86b9c4f2a5fa None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:54.100291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136a382-deb0-4bd4-9cb3-86b9c4f2a5fa None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:54.101167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136a382-deb0-4bd4-9cb3-86b9c4f2a5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:54.166075 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-8136a382-deb0-4bd4-9cb3-86b9c4f2a5fa None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:54.167310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8136a382-deb0-4bd4-9cb3-86b9c4f2a5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:54.185590 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:54.263443 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 21:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:54.299685 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d1093-e9b1-424b-94c0-664f340a4b1e None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:54.301264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e39d1093-e9b1-424b-94c0-664f340a4b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:54.407331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d1093-e9b1-424b-94c0-664f340a4b1e None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:54.474437 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:54.481826 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfe720-0a92-48ce-99a8-b05660f60136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:54.482481 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfe720-0a92-48ce-99a8-b05660f60136 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:54.482768 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfe720-0a92-48ce-99a8-b05660f60136 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:54.483049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfe720-0a92-48ce-99a8-b05660f60136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:54.484727 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-61dfe720-0a92-48ce-99a8-b05660f60136 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:54.594527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dfe720-0a92-48ce-99a8-b05660f60136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:54.595870 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dfe720-0a92-48ce-99a8-b05660f60136 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:54.614691 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 739/1467] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 21:59:56.688374 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca4657-29b4-4104-b564-96827a251864 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:56.689022 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca4657-29b4-4104-b564-96827a251864 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:56.689022 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca4657-29b4-4104-b564-96827a251864 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:56.690246 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 21:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 21:59:56.696542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc91e67-8d70-4cbd-90aa-3d89fce2fe6e None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:56.697022 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc91e67-8d70-4cbd-90aa-3d89fce2fe6e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:56.697148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc91e67-8d70-4cbd-90aa-3d89fce2fe6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:56.763089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-5dc91e67-8d70-4cbd-90aa-3d89fce2fe6e None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 21:59:56.764328 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5dc91e67-8d70-4cbd-90aa-3d89fce2fe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:56.813705 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:56.863319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 21:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 21:59:56.900912 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca4657-29b4-4104-b564-96827a251864 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 21:59:56.902465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-01ca4657-29b4-4104-b564-96827a251864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:57.009713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca4657-29b4-4104-b564-96827a251864 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 21:59:57.058834 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 21:59:57.085298 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45638f7-27ce-43f5-92b2-ea0c80451343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 21:59:57.085947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45638f7-27ce-43f5-92b2-ea0c80451343 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 21:59:57.086234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45638f7-27ce-43f5-92b2-ea0c80451343 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 21:59:57.086508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45638f7-27ce-43f5-92b2-ea0c80451343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 21:59:57.088160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b45638f7-27ce-43f5-92b2-ea0c80451343 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 21:59:57.200980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45638f7-27ce-43f5-92b2-ea0c80451343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 21:59:57.202443 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45638f7-27ce-43f5-92b2-ea0c80451343 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 21:59:57.221528 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 730/1470] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 21:59:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:00:06.861502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=22 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:00:07.219129 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=66 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:00:17.100119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fff57-56fd-4cdd-a184-8239b07c9af2 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:17.100696 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fff57-56fd-4cdd-a184-8239b07c9af2 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:17.100696 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fff57-56fd-4cdd-a184-8239b07c9af2 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:17.102107 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 741/1471] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 22:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:00:17.107266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3426e-6f36-43eb-9db4-b48f751cfb57 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:17.107680 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3426e-6f36-43eb-9db4-b48f751cfb57 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:17.107816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3426e-6f36-43eb-9db4-b48f751cfb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:17.171887 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-b4c3426e-6f36-43eb-9db4-b48f751cfb57 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:00:17.173125 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b4c3426e-6f36-43eb-9db4-b48f751cfb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:17.275989 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 731/1472] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 22:00:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:00:17.361340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05839ac-3372-40f4-a17e-8a87a0b70e6e None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:17.361635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05839ac-3372-40f4-a17e-8a87a0b70e6e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:17.361919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05839ac-3372-40f4-a17e-8a87a0b70e6e None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:17.363294 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 742/1473] 10.222.0.44 () {54 vars in 962 bytes} [Tue Jul 12 22:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:00:17.368736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c3832-fc33-4867-8d21-0da4cfa15778 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:17.369035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c3832-fc33-4867-8d21-0da4cfa15778 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:17.369318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c3832-fc33-4867-8d21-0da4cfa15778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:17.443345 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-218c3832-fc33-4867-8d21-0da4cfa15778 None None] MFA Rules not processed for user `c5c9ed6f9fd542069cf914003b636c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:00:17.445284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-218c3832-fc33-4867-8d21-0da4cfa15778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:17.465880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:00:17.520652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 732/1474] 10.222.0.44 () {58 vars in 1059 bytes} [Tue Jul 12 22:00:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:00:17.528226 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05839ac-3372-40f4-a17e-8a87a0b70e6e None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:00:17.529750 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c05839ac-3372-40f4-a17e-8a87a0b70e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:17.635309 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05839ac-3372-40f4-a17e-8a87a0b70e6e None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:00:17.708549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5e4df-ba61-48cd-b553-b5fe0757c5e2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5c9ed6f9fd542069cf914003b636c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:00:17.709224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5e4df-ba61-48cd-b553-b5fe0757c5e2 service glance] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:17.710118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5e4df-ba61-48cd-b553-b5fe0757c5e2 service glance] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:17.710539 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5e4df-ba61-48cd-b553-b5fe0757c5e2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:17.712196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-f9b5e4df-ba61-48cd-b553-b5fe0757c5e2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:17.767592 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:00:17.823223 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5e4df-ba61-48cd-b553-b5fe0757c5e2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:00:17.824570 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5e4df-ba61-48cd-b553-b5fe0757c5e2 service glance] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:00:17.843787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 743/1475] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:00:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:00:19.961615 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2e286-5f80-4d44-bba1-f7f2ef6c4d87 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:19.962219 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2e286-5f80-4d44-bba1-f7f2ef6c4d87 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:19.962219 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2e286-5f80-4d44-bba1-f7f2ef6c4d87 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:19.963546 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 733/1476] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 22:00:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:00:19.969002 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32165be2-d03f-4157-847d-0e00577a3eda None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:19.969466 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32165be2-d03f-4157-847d-0e00577a3eda None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:19.969575 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32165be2-d03f-4157-847d-0e00577a3eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:20.035003 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-32165be2-d03f-4157-847d-0e00577a3eda None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:00:20.036225 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-32165be2-d03f-4157-847d-0e00577a3eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:20.135221 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 744/1477] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 22:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:00:20.159058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2e286-5f80-4d44-bba1-f7f2ef6c4d87 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:00:20.160652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-dce2e286-5f80-4d44-bba1-f7f2ef6c4d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:20.286961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2e286-5f80-4d44-bba1-f7f2ef6c4d87 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:00:20.336010 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:00:20.380502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643bfc4a-5338-4764-bb4c-f1ba995d37eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:00:20.381200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643bfc4a-5338-4764-bb4c-f1ba995d37eb admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:20.382115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643bfc4a-5338-4764-bb4c-f1ba995d37eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:20.382397 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643bfc4a-5338-4764-bb4c-f1ba995d37eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:20.383137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643bfc4a-5338-4764-bb4c-f1ba995d37eb admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:00:20.385005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: WARNING py.warnings [None req-643bfc4a-5338-4764-bb4c-f1ba995d37eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 22:00:20.385005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: warnings.warn(msg) Jul 12 22:00:20.385005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: Jul 12 22:00:20.386188 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643bfc4a-5338-4764-bb4c-f1ba995d37eb admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:00:20.406169 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 734/1478] 10.222.0.44 () {56 vars in 1108 bytes} [Tue Jul 12 22:00:20 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:00:22.489510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1e5f6-58b4-4a0b-b2c0-13dbbbda7916 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:22.490028 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1e5f6-58b4-4a0b-b2c0-13dbbbda7916 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:22.490143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1e5f6-58b4-4a0b-b2c0-13dbbbda7916 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:22.492272 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 745/1479] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 22:00:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:00:22.497784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1bc45-16e8-47d1-a39b-7bdae0887a53 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:22.498219 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1bc45-16e8-47d1-a39b-7bdae0887a53 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:22.498333 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1bc45-16e8-47d1-a39b-7bdae0887a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:22.564001 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-70d1bc45-16e8-47d1-a39b-7bdae0887a53 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:00:22.565311 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-70d1bc45-16e8-47d1-a39b-7bdae0887a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:22.663466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 735/1480] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 22:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:00:22.706631 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1e5f6-58b4-4a0b-b2c0-13dbbbda7916 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:00:22.708243 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a6d1e5f6-58b4-4a0b-b2c0-13dbbbda7916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:22.718741 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:00:22.853060 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1e5f6-58b4-4a0b-b2c0-13dbbbda7916 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:00:22.935252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:00:22.970529 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1b0a0-e7e8-4ae3-96b2-f1f9a0be1b57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:00:22.971127 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1b0a0-e7e8-4ae3-96b2-f1f9a0be1b57 service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:22.971412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1b0a0-e7e8-4ae3-96b2-f1f9a0be1b57 service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:22.971693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1b0a0-e7e8-4ae3-96b2-f1f9a0be1b57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:22.973395 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-88e1b0a0-e7e8-4ae3-96b2-f1f9a0be1b57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:23.071373 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1b0a0-e7e8-4ae3-96b2-f1f9a0be1b57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:00:23.073041 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1b0a0-e7e8-4ae3-96b2-f1f9a0be1b57 service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:00:23.091475 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 746/1481] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:00:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:00:25.319534 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965ae1b-441b-4c9b-b66f-111ee50db9e0 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:25.320108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965ae1b-441b-4c9b-b66f-111ee50db9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:25.320108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965ae1b-441b-4c9b-b66f-111ee50db9e0 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:25.321482 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 736/1482] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 22:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:00:25.326693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194e652-be86-498a-b46c-6f4a70b7db90 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:25.327147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194e652-be86-498a-b46c-6f4a70b7db90 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:25.327248 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194e652-be86-498a-b46c-6f4a70b7db90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:25.391335 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-1194e652-be86-498a-b46c-6f4a70b7db90 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:00:25.392552 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-1194e652-be86-498a-b46c-6f4a70b7db90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:25.440949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:00:25.492736 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 747/1483] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 22:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:00:25.529286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965ae1b-441b-4c9b-b66f-111ee50db9e0 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:00:25.530845 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8965ae1b-441b-4c9b-b66f-111ee50db9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:25.554448 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:00:25.637475 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965ae1b-441b-4c9b-b66f-111ee50db9e0 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:00:25.720888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4ad76-6421-425e-89c1-cf09af796d26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:00:25.721542 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4ad76-6421-425e-89c1-cf09af796d26 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:25.721826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4ad76-6421-425e-89c1-cf09af796d26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:25.722103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4ad76-6421-425e-89c1-cf09af796d26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:25.723786 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-9be4ad76-6421-425e-89c1-cf09af796d26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:25.799313 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:00:25.835102 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4ad76-6421-425e-89c1-cf09af796d26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:00:25.836550 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4ad76-6421-425e-89c1-cf09af796d26 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:00:25.855316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 737/1484] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:00:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:00:27.975015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f59571-bb5c-4afd-82aa-12abe65c0856 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:27.975565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f59571-bb5c-4afd-82aa-12abe65c0856 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:27.975565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f59571-bb5c-4afd-82aa-12abe65c0856 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:27.977028 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 748/1485] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 22:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:00:27.982188 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be2308-6327-4643-a1c0-2c3fcdd71a06 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:27.982595 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be2308-6327-4643-a1c0-2c3fcdd71a06 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:27.982752 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be2308-6327-4643-a1c0-2c3fcdd71a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:28.046053 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-24be2308-6327-4643-a1c0-2c3fcdd71a06 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:00:28.047340 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-24be2308-6327-4643-a1c0-2c3fcdd71a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:28.144852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 738/1486] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 22:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:00:28.183419 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f59571-bb5c-4afd-82aa-12abe65c0856 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:00:28.185201 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-94f59571-bb5c-4afd-82aa-12abe65c0856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:28.290787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f59571-bb5c-4afd-82aa-12abe65c0856 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:00:28.362417 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:00:28.364358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcfbcd-3774-4a7f-ac7c-93c39445d001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:00:28.365220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcfbcd-3774-4a7f-ac7c-93c39445d001 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:28.366302 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcfbcd-3774-4a7f-ac7c-93c39445d001 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:28.366717 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcfbcd-3774-4a7f-ac7c-93c39445d001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:28.368577 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-07fcfbcd-3774-4a7f-ac7c-93c39445d001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:28.477019 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcfbcd-3774-4a7f-ac7c-93c39445d001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:00:28.478477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcfbcd-3774-4a7f-ac7c-93c39445d001 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:00:28.497299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 749/1487] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:00:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:00:30.657678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e80d4-b89e-41e2-b556-2560f2bd29d4 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:30.658241 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e80d4-b89e-41e2-b556-2560f2bd29d4 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:30.658241 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e80d4-b89e-41e2-b556-2560f2bd29d4 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:30.659604 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 739/1488] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 22:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:00:30.665816 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbbb69-14eb-4e14-8588-e6a1d8cfda22 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:30.666244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbbb69-14eb-4e14-8588-e6a1d8cfda22 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:30.666373 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbbb69-14eb-4e14-8588-e6a1d8cfda22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:30.732602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-d5dbbb69-14eb-4e14-8588-e6a1d8cfda22 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:00:30.733772 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d5dbbb69-14eb-4e14-8588-e6a1d8cfda22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:30.793314 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:00:30.839513 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 750/1489] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 22:00:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:00:30.879287 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e80d4-b89e-41e2-b556-2560f2bd29d4 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:00:30.880872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-420e80d4-b89e-41e2-b556-2560f2bd29d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:30.917677 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:00:30.986816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e80d4-b89e-41e2-b556-2560f2bd29d4 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:00:31.064131 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a63f50-63ab-4a4e-9a32-927c538f637d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:00:31.064872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a63f50-63ab-4a4e-9a32-927c538f637d service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:31.065172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a63f50-63ab-4a4e-9a32-927c538f637d service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:31.065454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a63f50-63ab-4a4e-9a32-927c538f637d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:31.067427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b8a63f50-63ab-4a4e-9a32-927c538f637d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:31.151619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:00:31.181026 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a63f50-63ab-4a4e-9a32-927c538f637d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:00:31.182481 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a63f50-63ab-4a4e-9a32-927c538f637d service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:00:31.201521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 740/1490] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:00:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:00:33.456213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5cebe-b8e5-4a95-828d-2e83cfa0042f None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:33.456814 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5cebe-b8e5-4a95-828d-2e83cfa0042f None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:33.456814 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5cebe-b8e5-4a95-828d-2e83cfa0042f None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:33.458249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 751/1491] 10.222.0.44 () {54 vars in 926 bytes} [Tue Jul 12 22:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:00:33.464171 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bdd5d-ef31-47eb-9ebb-7333739c3acf None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:33.464630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bdd5d-ef31-47eb-9ebb-7333739c3acf None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:33.464756 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bdd5d-ef31-47eb-9ebb-7333739c3acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:33.528043 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-ec4bdd5d-ef31-47eb-9ebb-7333739c3acf None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:00:33.529280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ec4bdd5d-ef31-47eb-9ebb-7333739c3acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:33.623874 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 741/1492] 10.222.0.44 () {58 vars in 1023 bytes} [Tue Jul 12 22:00:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:00:33.663275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5cebe-b8e5-4a95-828d-2e83cfa0042f None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:00:33.664861 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-83f5cebe-b8e5-4a95-828d-2e83cfa0042f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:33.908394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5cebe-b8e5-4a95-828d-2e83cfa0042f None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:00:33.984375 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6d6e1-e0f1-4051-b230-ee52feaa9871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:00:33.985060 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6d6e1-e0f1-4051-b230-ee52feaa9871 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:00:33.985343 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6d6e1-e0f1-4051-b230-ee52feaa9871 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:00:33.985619 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6d6e1-e0f1-4051-b230-ee52feaa9871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:00:33.987264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a2a6d6e1-e0f1-4051-b230-ee52feaa9871 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:00:33.995768 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:00:34.096087 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6d6e1-e0f1-4051-b230-ee52feaa9871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:00:34.097562 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6d6e1-e0f1-4051-b230-ee52feaa9871 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:00:34.115513 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 752/1493] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:00:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:00:43.622064 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=82 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:00:44.113185 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=50 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:02:01.410050 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58829c0b-b53d-41ad-8472-4c2f80308b53 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:02:01.411305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58829c0b-b53d-41ad-8472-4c2f80308b53 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:02:01.411305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58829c0b-b53d-41ad-8472-4c2f80308b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:02:01.487591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-58829c0b-b53d-41ad-8472-4c2f80308b53 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:02:01.488890 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-58829c0b-b53d-41ad-8472-4c2f80308b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:02:01.591960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 742/1494] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:02:01.600911 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a538116-a57c-4965-bc04-00410024a6f2 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:02:01.601440 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a538116-a57c-4965-bc04-00410024a6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:02:01.601440 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a538116-a57c-4965-bc04-00410024a6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:02:01.674008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-7a538116-a57c-4965-bc04-00410024a6f2 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:02:01.675127 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-7a538116-a57c-4965-bc04-00410024a6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:02:01.773756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 753/1495] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:02:01.778361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58829c0b-b53d-41ad-8472-4c2f80308b53 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:02:01.779571 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-58829c0b-b53d-41ad-8472-4c2f80308b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:02:01.791221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58829c0b-b53d-41ad-8472-4c2f80308b53 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:02:01.854581 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:02:01.888558 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec472e19-58ef-4c54-9982-706e4e6690ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:02:01.889250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec472e19-58ef-4c54-9982-706e4e6690ce admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:02:01.889532 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec472e19-58ef-4c54-9982-706e4e6690ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:02:01.889806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec472e19-58ef-4c54-9982-706e4e6690ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:02:01.890536 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec472e19-58ef-4c54-9982-706e4e6690ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:02:01.890839 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec472e19-58ef-4c54-9982-706e4e6690ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:02:01.891866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: WARNING py.warnings [None req-ec472e19-58ef-4c54-9982-706e4e6690ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 22:02:01.891866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: warnings.warn(msg) Jul 12 22:02:01.891866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: Jul 12 22:02:01.892773 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec472e19-58ef-4c54-9982-706e4e6690ce admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:02:01.905113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 743/1496] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:02:01.913995 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a538116-a57c-4965-bc04-00410024a6f2 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:02:01.923223 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a538116-a57c-4965-bc04-00410024a6f2 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:02:01.987412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:02:02.020781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122f5c4-96f7-482d-a3ca-296a30379320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:02:02.021439 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122f5c4-96f7-482d-a3ca-296a30379320 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:02:02.021717 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122f5c4-96f7-482d-a3ca-296a30379320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:02:02.021990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122f5c4-96f7-482d-a3ca-296a30379320 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:02:02.022680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122f5c4-96f7-482d-a3ca-296a30379320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:02:02.024404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: WARNING py.warnings [None req-5122f5c4-96f7-482d-a3ca-296a30379320 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 22:02:02.024404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: warnings.warn(msg) Jul 12 22:02:02.024404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: Jul 12 22:02:02.025627 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122f5c4-96f7-482d-a3ca-296a30379320 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:02:02.079502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 754/1497] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:02:01 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:02:02.083787 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec472e19-58ef-4c54-9982-706e4e6690ce admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:02:02.085350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ec472e19-58ef-4c54-9982-706e4e6690ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:02:02.119308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec472e19-58ef-4c54-9982-706e4e6690ce admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:02:02.215214 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ab13c-cb96-45fb-a1b0-c6959a0ee026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:02:02.215888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ab13c-cb96-45fb-a1b0-c6959a0ee026 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:02:02.216180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ab13c-cb96-45fb-a1b0-c6959a0ee026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:02:02.216513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ab13c-cb96-45fb-a1b0-c6959a0ee026 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:02:02.217200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ab13c-cb96-45fb-a1b0-c6959a0ee026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:02:02.218225 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: WARNING py.warnings [None req-db4ab13c-cb96-45fb-a1b0-c6959a0ee026 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 22:02:02.218225 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: warnings.warn(msg) Jul 12 22:02:02.218225 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: Jul 12 22:02:02.219434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ab13c-cb96-45fb-a1b0-c6959a0ee026 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:02:02.261135 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 744/1498] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:02:02 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:02:02.265315 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122f5c4-96f7-482d-a3ca-296a30379320 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:02:02.266842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5122f5c4-96f7-482d-a3ca-296a30379320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:02:02.331159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122f5c4-96f7-482d-a3ca-296a30379320 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:02:02.415181 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:02:02.427328 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286ff7b-1953-4335-8440-3446e45ec5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:02:02.428006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286ff7b-1953-4335-8440-3446e45ec5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:02:02.428294 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286ff7b-1953-4335-8440-3446e45ec5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:02:02.428567 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286ff7b-1953-4335-8440-3446e45ec5f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:02:02.429338 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286ff7b-1953-4335-8440-3446e45ec5f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:02:02.432017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: WARNING py.warnings [None req-2286ff7b-1953-4335-8440-3446e45ec5f3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 22:02:02.432017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: warnings.warn(msg) Jul 12 22:02:02.432017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: Jul 12 22:02:02.432945 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286ff7b-1953-4335-8440-3446e45ec5f3 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:02:02.448978 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 755/1499] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:02:02.454313 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ab13c-cb96-45fb-a1b0-c6959a0ee026 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:02:02.455858 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-db4ab13c-cb96-45fb-a1b0-c6959a0ee026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:02:02.467971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ab13c-cb96-45fb-a1b0-c6959a0ee026 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:02:02.493100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:02:02.571457 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2aa588-5766-41c3-b496-b3320bb94731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:02:02.572122 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aa588-5766-41c3-b496-b3320bb94731 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:02:02.572402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aa588-5766-41c3-b496-b3320bb94731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:02:02.572675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aa588-5766-41c3-b496-b3320bb94731 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:02:02.573429 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2aa588-5766-41c3-b496-b3320bb94731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:02:02.574446 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: WARNING py.warnings [None req-8f2aa588-5766-41c3-b496-b3320bb94731 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 22:02:02.574446 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: warnings.warn(msg) Jul 12 22:02:02.574446 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: Jul 12 22:02:02.575332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2aa588-5766-41c3-b496-b3320bb94731 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:02:02.588122 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 745/1500] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:02:02.592413 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286ff7b-1953-4335-8440-3446e45ec5f3 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:02:02.594022 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2286ff7b-1953-4335-8440-3446e45ec5f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:02:02.627130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286ff7b-1953-4335-8440-3446e45ec5f3 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:02:02.721984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9162334-54c4-4652-bcc4-8130ba52961f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:02:02.723435 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9162334-54c4-4652-bcc4-8130ba52961f admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:02:02.723878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9162334-54c4-4652-bcc4-8130ba52961f admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:02:02.724235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9162334-54c4-4652-bcc4-8130ba52961f admin admin] PATH_INFO: `/v3/projects/07520c4493e74ca0a6df1f86d9f581fc/users/4e10cb9d22c344da9503130a734d8e86/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:02:02.749031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9162334-54c4-4652-bcc4-8130ba52961f admin admin] RBAC: Authorizing `identity:create_grant(project_id=07520c4493e74ca0a6df1f86d9f581fc, user_id=4e10cb9d22c344da9503130a734d8e86, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:02:02.750069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: WARNING py.warnings [None req-f9162334-54c4-4652-bcc4-8130ba52961f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 22:02:02.750069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: warnings.warn(msg) Jul 12 22:02:02.750069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: Jul 12 22:02:02.752013 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9162334-54c4-4652-bcc4-8130ba52961f admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:02:02.776906 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 756/1501] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:02:02 2022] PUT /identity/v3/projects/07520c4493e74ca0a6df1f86d9f581fc/users/4e10cb9d22c344da9503130a734d8e86/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:02:02.789494 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee16d03-18d7-4dc7-8555-e26ae906b8f7 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:02:02.789786 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee16d03-18d7-4dc7-8555-e26ae906b8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:02:02.790072 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee16d03-18d7-4dc7-8555-e26ae906b8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:02:02.846155 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-3ee16d03-18d7-4dc7-8555-e26ae906b8f7 None None] MFA Rules not processed for user `4e10cb9d22c344da9503130a734d8e86`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:02:02.847361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-3ee16d03-18d7-4dc7-8555-e26ae906b8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:02:02.926974 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 746/1502] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:02:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:02:02.940313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702bc84-8f06-4d2c-9c57-a016076d6a96 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:02:02.940902 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702bc84-8f06-4d2c-9c57-a016076d6a96 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:02:02.941182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702bc84-8f06-4d2c-9c57-a016076d6a96 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:02:02.943496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 757/1503] 10.222.0.44 () {54 vars in 963 bytes} [Tue Jul 12 22:02:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:02:02.948476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4640b03-2c4b-487b-ad09-5e7c75330d0c None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:02:02.948798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4640b03-2c4b-487b-ad09-5e7c75330d0c None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:02:02.949095 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4640b03-2c4b-487b-ad09-5e7c75330d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:02:02.979414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:02:03.034625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-e4640b03-2c4b-487b-ad09-5e7c75330d0c None None] MFA Rules not processed for user `c53e1e801e5e40e794e4ccfff34ec988`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:02:03.035959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e4640b03-2c4b-487b-ad09-5e7c75330d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:02:03.112980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 747/1504] 10.222.0.44 () {58 vars in 1060 bytes} [Tue Jul 12 22:02:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:02:03.121383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702bc84-8f06-4d2c-9c57-a016076d6a96 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:02:03.122905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d702bc84-8f06-4d2c-9c57-a016076d6a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:02:03.172872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:02:03.230835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702bc84-8f06-4d2c-9c57-a016076d6a96 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:02:03.303851 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d025265-3d60-4d2c-a8e0-54c4ac6451da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:02:03.304498 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d025265-3d60-4d2c-a8e0-54c4ac6451da service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:02:03.304810 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d025265-3d60-4d2c-a8e0-54c4ac6451da service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:02:03.305091 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d025265-3d60-4d2c-a8e0-54c4ac6451da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:02:03.306761 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-3d025265-3d60-4d2c-a8e0-54c4ac6451da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:02:03.407115 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d025265-3d60-4d2c-a8e0-54c4ac6451da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:02:03.408554 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d025265-3d60-4d2c-a8e0-54c4ac6451da service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:02:03.412679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:02:03.428509 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 758/1505] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:02:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:02:03.470798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08409597-c5a1-4323-bfea-97f5c80bc834 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:02:03.471236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08409597-c5a1-4323-bfea-97f5c80bc834 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:02:03.471343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08409597-c5a1-4323-bfea-97f5c80bc834 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:02:03.472611 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 748/1506] 10.222.0.44 () {52 vars in 829 bytes} [Tue Jul 12 22:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:02:03.476787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d025265-3d60-4d2c-a8e0-54c4ac6451da service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:02:03.478008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-3d025265-3d60-4d2c-a8e0-54c4ac6451da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:02:03.578573 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d025265-3d60-4d2c-a8e0-54c4ac6451da service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:02:03.671707 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05697b-f721-4e63-bdbc-8e3de64712c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:02:03.672568 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05697b-f721-4e63-bdbc-8e3de64712c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:02:03.672882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05697b-f721-4e63-bdbc-8e3de64712c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:02:03.673168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05697b-f721-4e63-bdbc-8e3de64712c3 admin admin] PATH_INFO: `/v3/users/4e10cb9d22c344da9503130a734d8e86` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:02:03.685939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05697b-f721-4e63-bdbc-8e3de64712c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e10cb9d22c344da9503130a734d8e86)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:02:03.687093 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: WARNING py.warnings [None req-df05697b-f721-4e63-bdbc-8e3de64712c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 22:02:03.687093 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: warnings.warn(msg) Jul 12 22:02:03.687093 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: Jul 12 22:02:03.688317 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05697b-f721-4e63-bdbc-8e3de64712c3 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:02:03.698458 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:02:03.753812 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-df05697b-f721-4e63-bdbc-8e3de64712c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e10cb9d22c344da9503130a734d8e86'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:02:03.774816 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-df05697b-f721-4e63-bdbc-8e3de64712c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e10cb9d22c344da9503130a734d8e86'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:02:03.787547 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-df05697b-f721-4e63-bdbc-8e3de64712c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e10cb9d22c344da9503130a734d8e86'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:02:03.797280 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-df05697b-f721-4e63-bdbc-8e3de64712c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e10cb9d22c344da9503130a734d8e86'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:02:03.797761 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-df05697b-f721-4e63-bdbc-8e3de64712c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e10cb9d22c344da9503130a734d8e86'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:02:03.804322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 759/1507] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:02:03 2022] DELETE /identity/v3/users/4e10cb9d22c344da9503130a734d8e86 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:02:03.808917 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3bcfe-bc20-44f8-b238-19aeb2ea5116 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:02:03.809234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3bcfe-bc20-44f8-b238-19aeb2ea5116 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:02:03.809541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3bcfe-bc20-44f8-b238-19aeb2ea5116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:02:03.866897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-b1b3bcfe-bc20-44f8-b238-19aeb2ea5116 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:02:03.868049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b1b3bcfe-bc20-44f8-b238-19aeb2ea5116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:02:03.946831 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:02:03.960212 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 749/1508] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:02:03.969761 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05697b-f721-4e63-bdbc-8e3de64712c3 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:02:03.970987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-df05697b-f721-4e63-bdbc-8e3de64712c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:02:04.069613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05697b-f721-4e63-bdbc-8e3de64712c3 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:02:04.123877 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:02:04.144281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f0ede-d798-440b-8465-3d38953535c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:02:04.145010 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f0ede-d798-440b-8465-3d38953535c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:02:04.145295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f0ede-d798-440b-8465-3d38953535c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:02:04.145576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f0ede-d798-440b-8465-3d38953535c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:02:04.147224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ae2f0ede-d798-440b-8465-3d38953535c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:02:04.257127 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f0ede-d798-440b-8465-3d38953535c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:02:04.258574 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f0ede-d798-440b-8465-3d38953535c8 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:02:04.276759 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 760/1509] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:02:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:02:04.637767 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3bcfe-bc20-44f8-b238-19aeb2ea5116 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:02:04.639003 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b1b3bcfe-bc20-44f8-b238-19aeb2ea5116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:02:04.650989 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3bcfe-bc20-44f8-b238-19aeb2ea5116 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:02:04.760616 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfaa30-067a-422b-95d8-e3acd9a971b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:02:04.761516 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfaa30-067a-422b-95d8-e3acd9a971b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:02:04.761800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfaa30-067a-422b-95d8-e3acd9a971b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:02:04.762079 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfaa30-067a-422b-95d8-e3acd9a971b4 admin admin] PATH_INFO: `/v3/projects/07520c4493e74ca0a6df1f86d9f581fc` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:02:04.774882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbfaa30-067a-422b-95d8-e3acd9a971b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07520c4493e74ca0a6df1f86d9f581fc)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:02:04.775956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: WARNING py.warnings [None req-5cbfaa30-067a-422b-95d8-e3acd9a971b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 22:02:04.775956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: warnings.warn(msg) Jul 12 22:02:04.775956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: Jul 12 22:02:04.777191 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbfaa30-067a-422b-95d8-e3acd9a971b4 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:02:04.835486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-5cbfaa30-067a-422b-95d8-e3acd9a971b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07520c4493e74ca0a6df1f86d9f581fc', 'request_id': 'req-5cbfaa30-067a-422b-95d8-e3acd9a971b4'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:02:04.848433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-5cbfaa30-067a-422b-95d8-e3acd9a971b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07520c4493e74ca0a6df1f86d9f581fc', 'request_id': 'req-5cbfaa30-067a-422b-95d8-e3acd9a971b4'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:02:04.857440 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-5cbfaa30-067a-422b-95d8-e3acd9a971b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07520c4493e74ca0a6df1f86d9f581fc', 'request_id': 'req-5cbfaa30-067a-422b-95d8-e3acd9a971b4'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:02:04.881230 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-5cbfaa30-067a-422b-95d8-e3acd9a971b4 admin admin] The token cache is being invalidate because project 07520c4493e74ca0a6df1f86d9f581fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:02:04.881530 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-5cbfaa30-067a-422b-95d8-e3acd9a971b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:02:04.881973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-5cbfaa30-067a-422b-95d8-e3acd9a971b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:02:04.883498 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 750/1510] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:02:04 2022] DELETE /identity/v3/projects/07520c4493e74ca0a6df1f86d9f581fc => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:02:14.274467 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=62 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:02:14.878435 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:03:40.361823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d194308-0b6b-49e6-a346-c6ef5ce72dbd None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:40.362444 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d194308-0b6b-49e6-a346-c6ef5ce72dbd None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:40.362444 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d194308-0b6b-49e6-a346-c6ef5ce72dbd None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:40.363745 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 22:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:03:40.368512 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc2804-2aab-4ec5-9987-1ee29f90218e None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:40.369002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc2804-2aab-4ec5-9987-1ee29f90218e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:40.369110 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc2804-2aab-4ec5-9987-1ee29f90218e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:40.461362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-eccc2804-2aab-4ec5-9987-1ee29f90218e None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:03:40.463001 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-eccc2804-2aab-4ec5-9987-1ee29f90218e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:03:40.570653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 22:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:03:40.737584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d194308-0b6b-49e6-a346-c6ef5ce72dbd None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:03:40.739716 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8d194308-0b6b-49e6-a346-c6ef5ce72dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:03:40.910247 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d194308-0b6b-49e6-a346-c6ef5ce72dbd None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:03:40.916101 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699bb57-7a42-4817-a778-805f4b2630a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:03:40.916766 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699bb57-7a42-4817-a778-805f4b2630a3 service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:40.917053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699bb57-7a42-4817-a778-805f4b2630a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:40.917335 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699bb57-7a42-4817-a778-805f4b2630a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:40.925793 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699bb57-7a42-4817-a778-805f4b2630a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:03:40.927372 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699bb57-7a42-4817-a778-805f4b2630a3 service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:03:40.937526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 762/1513] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:03:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:03:41.848559 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42143447-970f-4080-ae09-a975514e6197 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:41.849134 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42143447-970f-4080-ae09-a975514e6197 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:41.849259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42143447-970f-4080-ae09-a975514e6197 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:41.850562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 22:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:03:41.855570 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868afc6-8500-441c-9cf9-614cd2f61757 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:41.856041 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868afc6-8500-441c-9cf9-614cd2f61757 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:41.856150 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868afc6-8500-441c-9cf9-614cd2f61757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:41.877721 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-d868afc6-8500-441c-9cf9-614cd2f61757 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:03:41.878948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d868afc6-8500-441c-9cf9-614cd2f61757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:03:41.887320 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 22:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:03:42.053438 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42143447-970f-4080-ae09-a975514e6197 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:03:42.087413 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42143447-970f-4080-ae09-a975514e6197 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:03:42.149269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be42cd-3e62-4b3a-acdf-83257aaaa2e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:03:42.149925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be42cd-3e62-4b3a-acdf-83257aaaa2e3 service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:42.150226 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be42cd-3e62-4b3a-acdf-83257aaaa2e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:42.150504 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be42cd-3e62-4b3a-acdf-83257aaaa2e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:42.160832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be42cd-3e62-4b3a-acdf-83257aaaa2e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:03:42.162461 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be42cd-3e62-4b3a-acdf-83257aaaa2e3 service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:03:42.173884 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 753/1516] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:03:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:03:42.393256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da426534-c5cb-4d13-bddb-cab7ae6d94cd None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:42.393542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da426534-c5cb-4d13-bddb-cab7ae6d94cd None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:42.393823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da426534-c5cb-4d13-bddb-cab7ae6d94cd None None] PATH_INFO: `/v3` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:42.395147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 764/1517] 10.222.0.27 () {54 vars in 925 bytes} [Tue Jul 12 22:03:42 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:03:42.401595 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705613a-3fd9-45da-b4ae-2db99a405aa4 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:42.401810 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705613a-3fd9-45da-b4ae-2db99a405aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:42.402087 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705613a-3fd9-45da-b4ae-2db99a405aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:42.463561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-e705613a-3fd9-45da-b4ae-2db99a405aa4 None None] MFA Rules not processed for user `a1f1ef52f8eb42a59a50a69873b9ffaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:03:42.464818 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e705613a-3fd9-45da-b4ae-2db99a405aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:03:42.485240 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:03:42.541065 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 754/1518] 10.222.0.27 () {58 vars in 1015 bytes} [Tue Jul 12 22:03:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:03:46.312159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dadb3-6eeb-40ee-a011-74910508346c None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:46.312762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dadb3-6eeb-40ee-a011-74910508346c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:46.312762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dadb3-6eeb-40ee-a011-74910508346c None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:46.314102 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 765/1519] 10.222.0.44 () {54 vars in 960 bytes} [Tue Jul 12 22:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 22:03:46.319404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af7ac2-4992-4a31-8e72-e4b1e17f777d None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:46.319840 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af7ac2-4992-4a31-8e72-e4b1e17f777d None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:46.319974 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af7ac2-4992-4a31-8e72-e4b1e17f777d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:46.341041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-89af7ac2-4992-4a31-8e72-e4b1e17f777d None None] MFA Rules not processed for user `a1f1ef52f8eb42a59a50a69873b9ffaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:03:46.342335 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-89af7ac2-4992-4a31-8e72-e4b1e17f777d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:03:46.351227 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 755/1520] 10.222.0.44 () {58 vars in 1057 bytes} [Tue Jul 12 22:03:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:03:46.359482 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9dadb3-6eeb-40ee-a011-74910508346c None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:03:46.373305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9dadb3-6eeb-40ee-a011-74910508346c None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:03:46.378263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a66f9-8643-4c7a-a687-b23b57ddfebf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f1ef52f8eb42a59a50a69873b9ffaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:03:46.378890 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a66f9-8643-4c7a-a687-b23b57ddfebf service placement] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:46.379191 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a66f9-8643-4c7a-a687-b23b57ddfebf service placement] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:46.379452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a66f9-8643-4c7a-a687-b23b57ddfebf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:46.387642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a66f9-8643-4c7a-a687-b23b57ddfebf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:03:46.389820 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a66f9-8643-4c7a-a687-b23b57ddfebf service placement] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:03:46.399391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 766/1521] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:03:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:03:53.490582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0faea-3e81-4304-95fc-b072c057f896 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:53.491155 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0faea-3e81-4304-95fc-b072c057f896 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:53.491155 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0faea-3e81-4304-95fc-b072c057f896 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:53.492520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 22:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:03:53.497797 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25f954-7f77-4d9c-ae9c-729752b36324 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:53.498218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25f954-7f77-4d9c-ae9c-729752b36324 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:53.498347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25f954-7f77-4d9c-ae9c-729752b36324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:53.516543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-4d25f954-7f77-4d9c-ae9c-729752b36324 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:03:53.517760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-4d25f954-7f77-4d9c-ae9c-729752b36324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:03:53.525487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 22:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:03:53.683000 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0faea-3e81-4304-95fc-b072c057f896 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:03:53.698142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0faea-3e81-4304-95fc-b072c057f896 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:03:53.704221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f33e4e-c2d9-44d5-bca0-b7065980d0cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:03:53.704949 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f33e4e-c2d9-44d5-bca0-b7065980d0cc service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:53.705236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f33e4e-c2d9-44d5-bca0-b7065980d0cc service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:53.705513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f33e4e-c2d9-44d5-bca0-b7065980d0cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:53.713962 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f33e4e-c2d9-44d5-bca0-b7065980d0cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:03:53.715543 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f33e4e-c2d9-44d5-bca0-b7065980d0cc service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:03:53.725010 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 757/1524] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:03:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:03:54.566506 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e593a6-4332-44a2-bb05-51dcfa2f8a75 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:54.567086 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e593a6-4332-44a2-bb05-51dcfa2f8a75 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:54.567086 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e593a6-4332-44a2-bb05-51dcfa2f8a75 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:54.568481 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 22:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:03:54.573573 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a23c9-ed1d-445a-b7dd-699fafd7be9c None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:54.573977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a23c9-ed1d-445a-b7dd-699fafd7be9c None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:54.574131 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a23c9-ed1d-445a-b7dd-699fafd7be9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:54.594028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-cd4a23c9-ed1d-445a-b7dd-699fafd7be9c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:03:54.595339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-cd4a23c9-ed1d-445a-b7dd-699fafd7be9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:03:54.607383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 22:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:03:54.762332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e593a6-4332-44a2-bb05-51dcfa2f8a75 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:03:54.776849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e593a6-4332-44a2-bb05-51dcfa2f8a75 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:03:54.782570 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d67de-cc95-4b48-8152-6d2b91ab4519 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:03:54.783234 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d67de-cc95-4b48-8152-6d2b91ab4519 service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:54.783516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d67de-cc95-4b48-8152-6d2b91ab4519 service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:54.783856 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d67de-cc95-4b48-8152-6d2b91ab4519 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:54.792052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d67de-cc95-4b48-8152-6d2b91ab4519 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:03:54.793670 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d67de-cc95-4b48-8152-6d2b91ab4519 service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:03:54.803982 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 769/1527] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:03:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:03:58.636734 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4f984-d0eb-4ad6-9bf4-1106d97843dd None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:58.637304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4f984-d0eb-4ad6-9bf4-1106d97843dd None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:58.637304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4f984-d0eb-4ad6-9bf4-1106d97843dd None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:58.639407 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 22:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:03:58.645500 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d492f15-1495-4100-81af-1d468fa8ce96 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:58.645973 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d492f15-1495-4100-81af-1d468fa8ce96 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:58.646094 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d492f15-1495-4100-81af-1d468fa8ce96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:58.664761 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-3d492f15-1495-4100-81af-1d468fa8ce96 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:03:58.665954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-3d492f15-1495-4100-81af-1d468fa8ce96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:03:58.673876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 22:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:03:58.748600 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4f984-d0eb-4ad6-9bf4-1106d97843dd None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:03:58.750366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d8c4f984-d0eb-4ad6-9bf4-1106d97843dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:03:58.832084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4f984-d0eb-4ad6-9bf4-1106d97843dd None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:03:58.836059 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6ba30-0919-4291-8641-7597c23bd9f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:03:58.836645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6ba30-0919-4291-8641-7597c23bd9f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:03:58.836957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6ba30-0919-4291-8641-7597c23bd9f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:03:58.837235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6ba30-0919-4291-8641-7597c23bd9f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:03:58.847438 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6ba30-0919-4291-8641-7597c23bd9f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:03:58.848956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6ba30-0919-4291-8641-7597c23bd9f1 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:03:58.859255 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 760/1530] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:03:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:04:00.901186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165ee8d-6e00-4ced-87ca-9352cb26f19f None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:04:00.901726 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165ee8d-6e00-4ced-87ca-9352cb26f19f None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:04:00.901726 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165ee8d-6e00-4ced-87ca-9352cb26f19f None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:04:00.903889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 22:04:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:04:00.909109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8262c6-10d5-491d-abfd-8aec0fa26e2c None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:04:00.909550 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8262c6-10d5-491d-abfd-8aec0fa26e2c None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:04:00.909655 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8262c6-10d5-491d-abfd-8aec0fa26e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:04:00.927746 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-1e8262c6-10d5-491d-abfd-8aec0fa26e2c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:04:00.928970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1e8262c6-10d5-491d-abfd-8aec0fa26e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:04:00.937591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 22:04:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:04:00.973390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165ee8d-6e00-4ced-87ca-9352cb26f19f None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:04:00.984051 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165ee8d-6e00-4ced-87ca-9352cb26f19f None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:04:00.987506 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf7abe-4e6a-4e40-939b-9cad4efa2f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:04:00.988134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf7abe-4e6a-4e40-939b-9cad4efa2f93 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:04:00.988416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf7abe-4e6a-4e40-939b-9cad4efa2f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:04:00.988693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf7abe-4e6a-4e40-939b-9cad4efa2f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:04:00.998120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf7abe-4e6a-4e40-939b-9cad4efa2f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:04:00.999539 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf7abe-4e6a-4e40-939b-9cad4efa2f93 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:04:01.008785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 772/1533] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:04:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:04:04.508948 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe30dc-34f7-4080-a02d-7d0392016a6d None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:04:04.509561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe30dc-34f7-4080-a02d-7d0392016a6d None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:04:04.509561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe30dc-34f7-4080-a02d-7d0392016a6d None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:04:04.510887 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 22:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:04:04.516217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5406b33-858b-45ca-a68d-88cf370292d4 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:04:04.516670 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5406b33-858b-45ca-a68d-88cf370292d4 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:04:04.516812 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5406b33-858b-45ca-a68d-88cf370292d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:04:04.522937 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=81 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:04:04.535495 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-e5406b33-858b-45ca-a68d-88cf370292d4 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:04:04.536730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e5406b33-858b-45ca-a68d-88cf370292d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:04:04.544919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 22:04:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:04:04.851547 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe30dc-34f7-4080-a02d-7d0392016a6d None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:04:04.853316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1bbe30dc-34f7-4080-a02d-7d0392016a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:04:04.919993 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:04:04.933270 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe30dc-34f7-4080-a02d-7d0392016a6d None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:04:04.937142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec604b5-41c0-47a6-9d93-43b1fcd5dc18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:04:04.937787 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec604b5-41c0-47a6-9d93-43b1fcd5dc18 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:04:04.938074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec604b5-41c0-47a6-9d93-43b1fcd5dc18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:04:04.938353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec604b5-41c0-47a6-9d93-43b1fcd5dc18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:04:04.947687 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec604b5-41c0-47a6-9d93-43b1fcd5dc18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:04:04.949181 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec604b5-41c0-47a6-9d93-43b1fcd5dc18 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:04:04.959179 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 763/1536] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:04:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:04:07.029770 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0f9e1-980b-4fa4-aea0-30f675a13a6a None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:04:07.030362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0f9e1-980b-4fa4-aea0-30f675a13a6a None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:04:07.030362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0f9e1-980b-4fa4-aea0-30f675a13a6a None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:04:07.031754 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 22:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:04:07.037278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf45b2b-b4fa-4fc9-8146-35bb19d7878f None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:04:07.037740 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf45b2b-b4fa-4fc9-8146-35bb19d7878f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:04:07.037882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf45b2b-b4fa-4fc9-8146-35bb19d7878f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:04:07.055793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-7cf45b2b-b4fa-4fc9-8146-35bb19d7878f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:04:07.057015 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-7cf45b2b-b4fa-4fc9-8146-35bb19d7878f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:04:07.065629 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 22:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:04:07.372509 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0f9e1-980b-4fa4-aea0-30f675a13a6a None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:04:07.374234 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-59e0f9e1-980b-4fa4-aea0-30f675a13a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:04:07.387004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0f9e1-980b-4fa4-aea0-30f675a13a6a None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:04:07.390488 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f5b7e-2205-4d52-a90e-1916dfb348e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:04:07.391116 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f5b7e-2205-4d52-a90e-1916dfb348e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:04:07.391396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f5b7e-2205-4d52-a90e-1916dfb348e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:04:07.391670 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f5b7e-2205-4d52-a90e-1916dfb348e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:04:07.400370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f5b7e-2205-4d52-a90e-1916dfb348e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:04:07.401838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f5b7e-2205-4d52-a90e-1916dfb348e8 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:04:07.411293 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 775/1539] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:04:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:04:17.050674 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=11 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:04:17.407210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=8 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:36.408700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231635a-7fae-4894-90e2-69a86d69cd6d None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:36.409876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231635a-7fae-4894-90e2-69a86d69cd6d None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:36.409876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231635a-7fae-4894-90e2-69a86d69cd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:36.439024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-3231635a-7fae-4894-90e2-69a86d69cd6d None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:36.440249 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-3231635a-7fae-4894-90e2-69a86d69cd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:36.452781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 765/1540] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:05:36.461437 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e939c-16b4-4d30-aca9-e40a574a7434 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:36.462038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e939c-16b4-4d30-aca9-e40a574a7434 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:36.462038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e939c-16b4-4d30-aca9-e40a574a7434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:36.489025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-0e7e939c-16b4-4d30-aca9-e40a574a7434 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:36.490715 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0e7e939c-16b4-4d30-aca9-e40a574a7434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:36.504424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 776/1541] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:36.509081 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231635a-7fae-4894-90e2-69a86d69cd6d None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:36.523880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231635a-7fae-4894-90e2-69a86d69cd6d None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:36.533310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9861d5-905f-43d7-8533-055c494a6b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:36.534013 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9861d5-905f-43d7-8533-055c494a6b67 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:36.534361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9861d5-905f-43d7-8533-055c494a6b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:36.534650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9861d5-905f-43d7-8533-055c494a6b67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:36.535407 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9861d5-905f-43d7-8533-055c494a6b67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:05:36.535724 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9861d5-905f-43d7-8533-055c494a6b67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:36.539392 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9861d5-905f-43d7-8533-055c494a6b67 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:36.561170 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 766/1542] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:05:36.569309 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e939c-16b4-4d30-aca9-e40a574a7434 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:36.586753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e939c-16b4-4d30-aca9-e40a574a7434 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:36.597231 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b95ef-db38-4fd7-8a66-d72bfcf0ac6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:36.598331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b95ef-db38-4fd7-8a66-d72bfcf0ac6b admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:36.598845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b95ef-db38-4fd7-8a66-d72bfcf0ac6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:36.599397 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b95ef-db38-4fd7-8a66-d72bfcf0ac6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:36.600372 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b95ef-db38-4fd7-8a66-d72bfcf0ac6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:36.602769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b95ef-db38-4fd7-8a66-d72bfcf0ac6b admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:36.648914 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 777/1543] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:36 2022] POST /identity/v3/projects => generated 365 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:36.653136 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9861d5-905f-43d7-8533-055c494a6b67 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:36.667591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9861d5-905f-43d7-8533-055c494a6b67 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:36.755811 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ac437-9485-4c92-ac76-894f7191ae65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:36.756513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ac437-9485-4c92-ac76-894f7191ae65 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:36.756878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ac437-9485-4c92-ac76-894f7191ae65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:36.757161 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ac437-9485-4c92-ac76-894f7191ae65 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:36.757864 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ac437-9485-4c92-ac76-894f7191ae65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:36.760509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ac437-9485-4c92-ac76-894f7191ae65 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:36.792881 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 767/1544] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:36 2022] POST /identity/v3/users => generated 283 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:36.797559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b95ef-db38-4fd7-8a66-d72bfcf0ac6b admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:36.813598 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b95ef-db38-4fd7-8a66-d72bfcf0ac6b admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:36.825637 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef22eda-97f6-4d2a-9cf2-ba433b119d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:36.826336 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef22eda-97f6-4d2a-9cf2-ba433b119d9e admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:36.826624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef22eda-97f6-4d2a-9cf2-ba433b119d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:36.826901 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef22eda-97f6-4d2a-9cf2-ba433b119d9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:36.827740 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef22eda-97f6-4d2a-9cf2-ba433b119d9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:36.829418 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef22eda-97f6-4d2a-9cf2-ba433b119d9e admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:36.846909 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 778/1545] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:36.851404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ac437-9485-4c92-ac76-894f7191ae65 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:36.851813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4504da-d4c5-4b41-ade3-3509f2e925f6 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:36.851937 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4504da-d4c5-4b41-ade3-3509f2e925f6 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:36.852215 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4504da-d4c5-4b41-ade3-3509f2e925f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:36.872614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ac437-9485-4c92-ac76-894f7191ae65 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:36.881444 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4d151-06c2-44b3-8e43-d6b568fd62f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:36.882468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4d151-06c2-44b3-8e43-d6b568fd62f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:36.882894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4d151-06c2-44b3-8e43-d6b568fd62f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:36.883189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4d151-06c2-44b3-8e43-d6b568fd62f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:36.883927 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4d151-06c2-44b3-8e43-d6b568fd62f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:36.892291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-8d4504da-d4c5-4b41-ade3-3509f2e925f6 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:36.892771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4d151-06c2-44b3-8e43-d6b568fd62f0 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:36.893604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8d4504da-d4c5-4b41-ade3-3509f2e925f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:36.914707 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 779/1546] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:36.919044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 768/1547] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:36.923403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4504da-d4c5-4b41-ade3-3509f2e925f6 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:36.925017 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0891ba-0b69-42e9-8951-d65a194f6594 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:36.925319 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0891ba-0b69-42e9-8951-d65a194f6594 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:36.925604 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0891ba-0b69-42e9-8951-d65a194f6594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:36.949171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4504da-d4c5-4b41-ade3-3509f2e925f6 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:36.958820 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-ce0891ba-0b69-42e9-8951-d65a194f6594 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:36.960048 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ce0891ba-0b69-42e9-8951-d65a194f6594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:36.960954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382ecbd-5507-41f2-a52d-4ed92c779248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:36.961859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382ecbd-5507-41f2-a52d-4ed92c779248 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:36.962140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382ecbd-5507-41f2-a52d-4ed92c779248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:36.962961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382ecbd-5507-41f2-a52d-4ed92c779248 admin admin] PATH_INFO: `/v3/projects/cde6ef220c0949119e7a9c559a6a93fb/users/518be0cad1d64ddabc1ab41ef3e456a6/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:36.995192 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 769/1548] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:36.995428 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382ecbd-5507-41f2-a52d-4ed92c779248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cde6ef220c0949119e7a9c559a6a93fb, user_id=518be0cad1d64ddabc1ab41ef3e456a6, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:36.996540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e8cad-4fce-4d2b-98d3-e3dbda424d48 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:36.996856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e8cad-4fce-4d2b-98d3-e3dbda424d48 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:36.997146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e8cad-4fce-4d2b-98d3-e3dbda424d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:36.998279 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382ecbd-5507-41f2-a52d-4ed92c779248 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.028885 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 780/1549] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:36 2022] PUT /identity/v3/projects/cde6ef220c0949119e7a9c559a6a93fb/users/518be0cad1d64ddabc1ab41ef3e456a6/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:37.029938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382ecbd-5507-41f2-a52d-4ed92c779248 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.037724 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-1d8e8cad-4fce-4d2b-98d3-e3dbda424d48 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:37.038971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1d8e8cad-4fce-4d2b-98d3-e3dbda424d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:37.143564 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 770/1550] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:37.145132 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe647e-4562-4913-be3a-ee913549c7ed None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.145429 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe647e-4562-4913-be3a-ee913549c7ed None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.145717 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe647e-4562-4913-be3a-ee913549c7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.169520 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382ecbd-5507-41f2-a52d-4ed92c779248 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.174797 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e028436-bf67-4f94-bac9-789a093032c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.175479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e028436-bf67-4f94-bac9-789a093032c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.175763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e028436-bf67-4f94-bac9-789a093032c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.176725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e028436-bf67-4f94-bac9-789a093032c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.177134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e028436-bf67-4f94-bac9-789a093032c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:05:37.177451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e028436-bf67-4f94-bac9-789a093032c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:37.178146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-7ffe647e-4562-4913-be3a-ee913549c7ed None None] MFA Rules not processed for user `518be0cad1d64ddabc1ab41ef3e456a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:37.178616 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: WARNING py.warnings [None req-3e028436-bf67-4f94-bac9-789a093032c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 22:05:37.178616 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: warnings.warn(msg) Jul 12 22:05:37.178616 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: Jul 12 22:05:37.179512 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e028436-bf67-4f94-bac9-789a093032c4 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.180083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-7ffe647e-4562-4913-be3a-ee913549c7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:37.192940 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 781/1551] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:05:37.194263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f3b61-e72e-416e-b8f8-bf5a33b327e2 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.194807 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f3b61-e72e-416e-b8f8-bf5a33b327e2 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.195092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f3b61-e72e-416e-b8f8-bf5a33b327e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.215048 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-845f3b61-e72e-416e-b8f8-bf5a33b327e2 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:37.216268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-845f3b61-e72e-416e-b8f8-bf5a33b327e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:37.225492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 782/1552] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:37.226929 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d7744-dc42-4dc4-a0f4-f6d6a155f7ac None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.227124 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d7744-dc42-4dc4-a0f4-f6d6a155f7ac None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.227408 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d7744-dc42-4dc4-a0f4-f6d6a155f7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.239675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:37.255170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-674d7744-dc42-4dc4-a0f4-f6d6a155f7ac None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:37.256822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-674d7744-dc42-4dc4-a0f4-f6d6a155f7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:37.265809 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 783/1553] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:37.267687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7244b5-d76b-4b8b-997b-acd567906775 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.268002 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7244b5-d76b-4b8b-997b-acd567906775 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.268255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7244b5-d76b-4b8b-997b-acd567906775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.277890 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 771/1554] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:37.279562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002935a-f982-4596-a1bc-f8444cf3f60f None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.279852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002935a-f982-4596-a1bc-f8444cf3f60f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.280132 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002935a-f982-4596-a1bc-f8444cf3f60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.287138 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-ff7244b5-d76b-4b8b-997b-acd567906775 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:37.288345 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ff7244b5-d76b-4b8b-997b-acd567906775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:37.297214 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 784/1555] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:37.298021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7244b5-d76b-4b8b-997b-acd567906775 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.301043 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-6002935a-f982-4596-a1bc-f8444cf3f60f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:37.302265 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6002935a-f982-4596-a1bc-f8444cf3f60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:37.308366 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7244b5-d76b-4b8b-997b-acd567906775 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.311152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 772/1556] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:37.312386 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b903a-0710-4ac4-bba5-44d2bbe8757b None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.312679 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b903a-0710-4ac4-bba5-44d2bbe8757b None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.312988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b903a-0710-4ac4-bba5-44d2bbe8757b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.313133 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496de100-aee8-4f79-afdb-0ebc73ea81dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.313693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496de100-aee8-4f79-afdb-0ebc73ea81dd admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.313981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496de100-aee8-4f79-afdb-0ebc73ea81dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.314257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496de100-aee8-4f79-afdb-0ebc73ea81dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.314945 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496de100-aee8-4f79-afdb-0ebc73ea81dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:37.317023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496de100-aee8-4f79-afdb-0ebc73ea81dd admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.331854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-ba6b903a-0710-4ac4-bba5-44d2bbe8757b None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:37.333180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ba6b903a-0710-4ac4-bba5-44d2bbe8757b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:37.342055 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 773/1557] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:37.343428 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74b80a-5147-4f19-9a86-93f26342d3f8 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.343728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74b80a-5147-4f19-9a86-93f26342d3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.344010 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74b80a-5147-4f19-9a86-93f26342d3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.344532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 785/1558] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:37.345772 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c0046-d7aa-4ff2-8a45-9e6451a0182e None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.346069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c0046-d7aa-4ff2-8a45-9e6451a0182e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.346373 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c0046-d7aa-4ff2-8a45-9e6451a0182e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.362822 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-bd74b80a-5147-4f19-9a86-93f26342d3f8 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:37.364222 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-bd74b80a-5147-4f19-9a86-93f26342d3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:37.366730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-167c0046-d7aa-4ff2-8a45-9e6451a0182e None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:37.367889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-167c0046-d7aa-4ff2-8a45-9e6451a0182e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:37.396740 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:37.425064 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 774/1559] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:37.426367 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000661e5-3072-404c-bffa-2733671417ee None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.426670 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000661e5-3072-404c-bffa-2733671417ee None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.426949 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000661e5-3072-404c-bffa-2733671417ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.437525 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 786/1560] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:37.438372 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c0046-d7aa-4ff2-8a45-9e6451a0182e None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.447259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-000661e5-3072-404c-bffa-2733671417ee None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:37.448227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c0046-d7aa-4ff2-8a45-9e6451a0182e None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.448508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-000661e5-3072-404c-bffa-2733671417ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:37.452913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99029d6f-5643-4c13-b9c0-b7529cc95c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.453518 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99029d6f-5643-4c13-b9c0-b7529cc95c35 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.453807 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99029d6f-5643-4c13-b9c0-b7529cc95c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.454153 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99029d6f-5643-4c13-b9c0-b7529cc95c35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.454918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99029d6f-5643-4c13-b9c0-b7529cc95c35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:37.456424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 775/1561] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:37.456781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99029d6f-5643-4c13-b9c0-b7529cc95c35 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.457676 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b6e07-f5a9-4a66-9601-40ddbae0b443 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.457962 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b6e07-f5a9-4a66-9601-40ddbae0b443 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.458240 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b6e07-f5a9-4a66-9601-40ddbae0b443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.477151 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-cc5b6e07-f5a9-4a66-9601-40ddbae0b443 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:37.479506 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-cc5b6e07-f5a9-4a66-9601-40ddbae0b443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:37.485879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 787/1562] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:37.486718 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99029d6f-5643-4c13-b9c0-b7529cc95c35 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.490170 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 776/1563] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:37.491006 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b6e07-f5a9-4a66-9601-40ddbae0b443 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.500853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b6e07-f5a9-4a66-9601-40ddbae0b443 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.550100 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99029d6f-5643-4c13-b9c0-b7529cc95c35 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.555064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8d19c-38bd-4fbc-a602-cb7377f1b5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.555728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8d19c-38bd-4fbc-a602-cb7377f1b5ed admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.556012 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8d19c-38bd-4fbc-a602-cb7377f1b5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.556427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8d19c-38bd-4fbc-a602-cb7377f1b5ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.557196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8d19c-38bd-4fbc-a602-cb7377f1b5ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:05:37.557509 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8d19c-38bd-4fbc-a602-cb7377f1b5ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:37.559139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8d19c-38bd-4fbc-a602-cb7377f1b5ed admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.560628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12252f32-94c7-4069-a192-52c8d924652b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.561772 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12252f32-94c7-4069-a192-52c8d924652b admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.561772 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12252f32-94c7-4069-a192-52c8d924652b admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.561974 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12252f32-94c7-4069-a192-52c8d924652b admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.562570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12252f32-94c7-4069-a192-52c8d924652b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:37.564476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12252f32-94c7-4069-a192-52c8d924652b admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.571199 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 788/1564] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:05:37.572759 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8d19c-38bd-4fbc-a602-cb7377f1b5ed admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.584607 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8d19c-38bd-4fbc-a602-cb7377f1b5ed admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.588829 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 777/1565] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:37.590166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7e347-aaba-49f0-8a6d-cd8a310a9f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.590401 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63cbfe-8211-4f46-82b1-5168ad3357ee None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.590581 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63cbfe-8211-4f46-82b1-5168ad3357ee None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.590826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63cbfe-8211-4f46-82b1-5168ad3357ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.590941 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7e347-aaba-49f0-8a6d-cd8a310a9f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.591116 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7e347-aaba-49f0-8a6d-cd8a310a9f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.591372 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7e347-aaba-49f0-8a6d-cd8a310a9f92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.592344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7e347-aaba-49f0-8a6d-cd8a310a9f92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:05:37.592657 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7e347-aaba-49f0-8a6d-cd8a310a9f92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:37.594287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7e347-aaba-49f0-8a6d-cd8a310a9f92 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.607827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 789/1566] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:05:37.609011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b59ed6-2382-4494-a9a8-4395e532f53c None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.609299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b59ed6-2382-4494-a9a8-4395e532f53c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.609432 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-7a63cbfe-8211-4f46-82b1-5168ad3357ee None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:37.609603 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b59ed6-2382-4494-a9a8-4395e532f53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.610536 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-7a63cbfe-8211-4f46-82b1-5168ad3357ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:37.619560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 778/1567] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:37.620625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63cbfe-8211-4f46-82b1-5168ad3357ee None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.630996 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63cbfe-8211-4f46-82b1-5168ad3357ee None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.631255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-82b59ed6-2382-4494-a9a8-4395e532f53c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:37.632255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-82b59ed6-2382-4494-a9a8-4395e532f53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:37.635784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedccf1-92c5-4461-9c22-e6528bb420ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.636535 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedccf1-92c5-4461-9c22-e6528bb420ea admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.636912 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedccf1-92c5-4461-9c22-e6528bb420ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.637219 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedccf1-92c5-4461-9c22-e6528bb420ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.637976 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cedccf1-92c5-4461-9c22-e6528bb420ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:05:37.638281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cedccf1-92c5-4461-9c22-e6528bb420ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:37.639931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cedccf1-92c5-4461-9c22-e6528bb420ea admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.641359 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 790/1568] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:37.641939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b59ed6-2382-4494-a9a8-4395e532f53c None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.651583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b59ed6-2382-4494-a9a8-4395e532f53c None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.653819 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 779/1569] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:05:37.654719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedccf1-92c5-4461-9c22-e6528bb420ea admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.656275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04a4f1-be60-4fc8-b5fc-74f2c8890261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.656963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04a4f1-be60-4fc8-b5fc-74f2c8890261 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.657251 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04a4f1-be60-4fc8-b5fc-74f2c8890261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.657530 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04a4f1-be60-4fc8-b5fc-74f2c8890261 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.658216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04a4f1-be60-4fc8-b5fc-74f2c8890261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:37.659240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: WARNING py.warnings [None req-4c04a4f1-be60-4fc8-b5fc-74f2c8890261 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 22:05:37.659240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: warnings.warn(msg) Jul 12 22:05:37.659240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: Jul 12 22:05:37.660442 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04a4f1-be60-4fc8-b5fc-74f2c8890261 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.666411 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedccf1-92c5-4461-9c22-e6528bb420ea admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.672935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df973e-d2e6-4fd8-8ca5-c5c237425232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.673640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df973e-d2e6-4fd8-8ca5-c5c237425232 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.673996 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df973e-d2e6-4fd8-8ca5-c5c237425232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.674276 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df973e-d2e6-4fd8-8ca5-c5c237425232 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.675018 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df973e-d2e6-4fd8-8ca5-c5c237425232 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:05:37.675345 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df973e-d2e6-4fd8-8ca5-c5c237425232 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:37.677108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df973e-d2e6-4fd8-8ca5-c5c237425232 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.685012 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 791/1570] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/users => generated 289 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:37.685012 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04a4f1-be60-4fc8-b5fc-74f2c8890261 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.695237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 780/1571] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:05:37.695237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df973e-d2e6-4fd8-8ca5-c5c237425232 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.699950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04a4f1-be60-4fc8-b5fc-74f2c8890261 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.704002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df973e-d2e6-4fd8-8ca5-c5c237425232 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.705031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8c8de-373e-49a3-aba0-1a1ef0c6f0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.705693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8c8de-373e-49a3-aba0-1a1ef0c6f0ab admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.705980 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8c8de-373e-49a3-aba0-1a1ef0c6f0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.706257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8c8de-373e-49a3-aba0-1a1ef0c6f0ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.706948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8c8de-373e-49a3-aba0-1a1ef0c6f0ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:37.708836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd82fb-4789-4ced-b0e5-7c3811186a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.709026 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8c8de-373e-49a3-aba0-1a1ef0c6f0ab admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.709520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd82fb-4789-4ced-b0e5-7c3811186a55 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.709798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd82fb-4789-4ced-b0e5-7c3811186a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.710073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd82fb-4789-4ced-b0e5-7c3811186a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.710793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd82fb-4789-4ced-b0e5-7c3811186a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:37.712920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd82fb-4789-4ced-b0e5-7c3811186a55 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.723733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 781/1572] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:37.724616 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd82fb-4789-4ced-b0e5-7c3811186a55 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.731138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:37.736728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd82fb-4789-4ced-b0e5-7c3811186a55 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.742359 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 792/1573] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:37.743199 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8c8de-373e-49a3-aba0-1a1ef0c6f0ab admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.747731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceaa4f1-e55e-4763-b1ee-a0c7bd507e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.748820 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaa4f1-e55e-4763-b1ee-a0c7bd507e75 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.749254 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaa4f1-e55e-4763-b1ee-a0c7bd507e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.749776 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaa4f1-e55e-4763-b1ee-a0c7bd507e75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.750565 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceaa4f1-e55e-4763-b1ee-a0c7bd507e75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:37.751700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: WARNING py.warnings [None req-5ceaa4f1-e55e-4763-b1ee-a0c7bd507e75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 22:05:37.751700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: warnings.warn(msg) Jul 12 22:05:37.751700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: Jul 12 22:05:37.752758 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceaa4f1-e55e-4763-b1ee-a0c7bd507e75 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.781561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 782/1574] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:37.782559 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceaa4f1-e55e-4763-b1ee-a0c7bd507e75 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.796609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:37.813404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8c8de-373e-49a3-aba0-1a1ef0c6f0ab admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.845749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceaa4f1-e55e-4763-b1ee-a0c7bd507e75 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.851031 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881bbc2-3e57-49b9-afb9-f05a21e54a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.851767 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881bbc2-3e57-49b9-afb9-f05a21e54a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.852117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881bbc2-3e57-49b9-afb9-f05a21e54a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.852488 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881bbc2-3e57-49b9-afb9-f05a21e54a4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.853894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881bbc2-3e57-49b9-afb9-f05a21e54a4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:05:37.854280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881bbc2-3e57-49b9-afb9-f05a21e54a4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:37.856007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881bbc2-3e57-49b9-afb9-f05a21e54a4d admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.867551 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 783/1575] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:05:37.868349 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881bbc2-3e57-49b9-afb9-f05a21e54a4d admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.874112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe9727-2e46-499c-87c0-19a3bfa05a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.874795 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe9727-2e46-499c-87c0-19a3bfa05a80 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.875086 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe9727-2e46-499c-87c0-19a3bfa05a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.875487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe9727-2e46-499c-87c0-19a3bfa05a80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.876315 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe9727-2e46-499c-87c0-19a3bfa05a80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:05:37.876703 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe9727-2e46-499c-87c0-19a3bfa05a80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:37.878442 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe9727-2e46-499c-87c0-19a3bfa05a80 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.884311 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881bbc2-3e57-49b9-afb9-f05a21e54a4d admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.889463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677974e-2be9-42df-b445-9ba862cc84b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.890122 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677974e-2be9-42df-b445-9ba862cc84b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.890404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677974e-2be9-42df-b445-9ba862cc84b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.890680 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677974e-2be9-42df-b445-9ba862cc84b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.891364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 793/1576] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:05:37.891543 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677974e-2be9-42df-b445-9ba862cc84b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:37.892200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe9727-2e46-499c-87c0-19a3bfa05a80 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.893571 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677974e-2be9-42df-b445-9ba862cc84b4 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.904217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe9727-2e46-499c-87c0-19a3bfa05a80 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.908881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af848cc-99aa-49bf-9614-f79e72fbd1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.909543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af848cc-99aa-49bf-9614-f79e72fbd1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.909836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af848cc-99aa-49bf-9614-f79e72fbd1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.910117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af848cc-99aa-49bf-9614-f79e72fbd1c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.910871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af848cc-99aa-49bf-9614-f79e72fbd1c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:37.912541 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af848cc-99aa-49bf-9614-f79e72fbd1c4 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.921836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 784/1577] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:37.922714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677974e-2be9-42df-b445-9ba862cc84b4 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.924624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 794/1578] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:37.925458 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af848cc-99aa-49bf-9614-f79e72fbd1c4 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:37.935671 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af848cc-99aa-49bf-9614-f79e72fbd1c4 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.989819 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677974e-2be9-42df-b445-9ba862cc84b4 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:37.994956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce70be-0549-42ad-af3e-733e63d5af66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.995692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce70be-0549-42ad-af3e-733e63d5af66 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.996008 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce70be-0549-42ad-af3e-733e63d5af66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.996311 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce70be-0549-42ad-af3e-733e63d5af66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:37.997079 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce70be-0549-42ad-af3e-733e63d5af66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:37.998056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c9581-0997-4d7c-b788-13b410aafd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:37.998723 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c9581-0997-4d7c-b788-13b410aafd9b admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:37.999007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c9581-0997-4d7c-b788-13b410aafd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:37.999128 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce70be-0549-42ad-af3e-733e63d5af66 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:37.999286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c9581-0997-4d7c-b788-13b410aafd9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.000005 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c9581-0997-4d7c-b788-13b410aafd9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.001627 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c9581-0997-4d7c-b788-13b410aafd9b admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.016145 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 795/1579] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:38.017169 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c9581-0997-4d7c-b788-13b410aafd9b admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.030921 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 785/1580] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:37 2022] POST /identity/v3/projects => generated 389 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:38.030921 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce70be-0549-42ad-af3e-733e63d5af66 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.035286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c9581-0997-4d7c-b788-13b410aafd9b admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.076015 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:38.096076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340727f-cbb1-471b-9b1e-fbf90d0a314f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.096854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340727f-cbb1-471b-9b1e-fbf90d0a314f admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.097184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340727f-cbb1-471b-9b1e-fbf90d0a314f admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.097487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340727f-cbb1-471b-9b1e-fbf90d0a314f admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.098233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b340727f-cbb1-471b-9b1e-fbf90d0a314f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.099030 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce70be-0549-42ad-af3e-733e63d5af66 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.100208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b340727f-cbb1-471b-9b1e-fbf90d0a314f admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.104496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d1c06-38c6-4e3e-9bbb-89ef8006b5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.105202 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d1c06-38c6-4e3e-9bbb-89ef8006b5d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.105485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d1c06-38c6-4e3e-9bbb-89ef8006b5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.105764 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d1c06-38c6-4e3e-9bbb-89ef8006b5d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.106457 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d1c06-38c6-4e3e-9bbb-89ef8006b5d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.108375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d1c06-38c6-4e3e-9bbb-89ef8006b5d6 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.123564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 796/1581] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:38 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:38.124195 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340727f-cbb1-471b-9b1e-fbf90d0a314f admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.129159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:38.131490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 786/1582] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:38 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:38.132334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d1c06-38c6-4e3e-9bbb-89ef8006b5d6 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.134854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340727f-cbb1-471b-9b1e-fbf90d0a314f admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.140830 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8869149-c8c4-48b0-b810-2faedc16f70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.141508 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8869149-c8c4-48b0-b810-2faedc16f70f admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.142064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8869149-c8c4-48b0-b810-2faedc16f70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.142429 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8869149-c8c4-48b0-b810-2faedc16f70f admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.142785 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d1c06-38c6-4e3e-9bbb-89ef8006b5d6 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.143403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8869149-c8c4-48b0-b810-2faedc16f70f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.145410 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8869149-c8c4-48b0-b810-2faedc16f70f admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.147611 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5df2d-a497-47d6-ad15-65cd3323afa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.148257 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5df2d-a497-47d6-ad15-65cd3323afa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.148538 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5df2d-a497-47d6-ad15-65cd3323afa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.148846 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5df2d-a497-47d6-ad15-65cd3323afa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.149531 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5df2d-a497-47d6-ad15-65cd3323afa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.151560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5df2d-a497-47d6-ad15-65cd3323afa8 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.179887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 797/1583] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:38 2022] POST /identity/v3/projects => generated 405 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:38.180074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 787/1584] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:38 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:38.180074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5df2d-a497-47d6-ad15-65cd3323afa8 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.180898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8869149-c8c4-48b0-b810-2faedc16f70f admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.245414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8869149-c8c4-48b0-b810-2faedc16f70f admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.245637 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5df2d-a497-47d6-ad15-65cd3323afa8 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.250550 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ec0f4-9ff6-4fb1-a4ed-605915ac7195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.250721 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba55f9-2d83-45d6-a137-3e02805fbd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.251393 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba55f9-2d83-45d6-a137-3e02805fbd91 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.251493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ec0f4-9ff6-4fb1-a4ed-605915ac7195 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.251682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba55f9-2d83-45d6-a137-3e02805fbd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.251782 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ec0f4-9ff6-4fb1-a4ed-605915ac7195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.251963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba55f9-2d83-45d6-a137-3e02805fbd91 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.252060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ec0f4-9ff6-4fb1-a4ed-605915ac7195 admin admin] PATH_INFO: `/v3/projects/453ee4bddca64aa5b4fd56447be26046/users/41b6585328bc4134a67820b5114c19c3/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.252657 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba55f9-2d83-45d6-a137-3e02805fbd91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.254678 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba55f9-2d83-45d6-a137-3e02805fbd91 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.266849 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ec0f4-9ff6-4fb1-a4ed-605915ac7195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=453ee4bddca64aa5b4fd56447be26046, user_id=41b6585328bc4134a67820b5114c19c3, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.268105 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: WARNING py.warnings [None req-a24ec0f4-9ff6-4fb1-a4ed-605915ac7195 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 22:05:38.268105 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: warnings.warn(msg) Jul 12 22:05:38.268105 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: Jul 12 22:05:38.270179 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ec0f4-9ff6-4fb1-a4ed-605915ac7195 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.275400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 798/1585] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:38 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:38.276375 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba55f9-2d83-45d6-a137-3e02805fbd91 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.286870 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba55f9-2d83-45d6-a137-3e02805fbd91 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.287331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 788/1586] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:38 2022] PUT /identity/v3/projects/453ee4bddca64aa5b4fd56447be26046/users/41b6585328bc4134a67820b5114c19c3/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:38.288284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ec0f4-9ff6-4fb1-a4ed-605915ac7195 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.355278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ec0f4-9ff6-4fb1-a4ed-605915ac7195 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.356105 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e3663-c0d4-4910-8a2b-3de3f3cd502f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.357158 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e3663-c0d4-4910-8a2b-3de3f3cd502f admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.357504 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e3663-c0d4-4910-8a2b-3de3f3cd502f admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.357829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e3663-c0d4-4910-8a2b-3de3f3cd502f admin admin] PATH_INFO: `/v3/projects/23c82522098348f589785650ba590b55/users/e7d835605f074ba885baffa7d023f943/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.360190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f1774-0dc0-4ad7-8a72-35a8ed1fdda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.360889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f1774-0dc0-4ad7-8a72-35a8ed1fdda3 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.361186 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f1774-0dc0-4ad7-8a72-35a8ed1fdda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.361462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f1774-0dc0-4ad7-8a72-35a8ed1fdda3 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.362153 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f1774-0dc0-4ad7-8a72-35a8ed1fdda3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.364095 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f1774-0dc0-4ad7-8a72-35a8ed1fdda3 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.373665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e3663-c0d4-4910-8a2b-3de3f3cd502f admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c82522098348f589785650ba590b55, user_id=e7d835605f074ba885baffa7d023f943, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.376524 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e3663-c0d4-4910-8a2b-3de3f3cd502f admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.386500 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 789/1587] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:38 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:38.387870 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f1774-0dc0-4ad7-8a72-35a8ed1fdda3 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.391091 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 799/1588] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:38 2022] PUT /identity/v3/projects/23c82522098348f589785650ba590b55/users/e7d835605f074ba885baffa7d023f943/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:38.391914 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e3663-c0d4-4910-8a2b-3de3f3cd502f admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.398542 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f1774-0dc0-4ad7-8a72-35a8ed1fdda3 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.410965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:38.445908 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:38.457085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e3663-c0d4-4910-8a2b-3de3f3cd502f admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.459335 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60b519-f593-4f07-9f33-1c48519cbaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.460046 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60b519-f593-4f07-9f33-1c48519cbaf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.460367 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60b519-f593-4f07-9f33-1c48519cbaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.460670 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60b519-f593-4f07-9f33-1c48519cbaf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.461484 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60b519-f593-4f07-9f33-1c48519cbaf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.461840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734fa36-b85d-4d06-975b-792045b28873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.463053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734fa36-b85d-4d06-975b-792045b28873 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.463330 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734fa36-b85d-4d06-975b-792045b28873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.463603 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734fa36-b85d-4d06-975b-792045b28873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.463736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60b519-f593-4f07-9f33-1c48519cbaf3 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.464326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734fa36-b85d-4d06-975b-792045b28873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.466020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734fa36-b85d-4d06-975b-792045b28873 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.473713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 790/1589] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:38.474509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60b519-f593-4f07-9f33-1c48519cbaf3 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.476085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 800/1590] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:38.477614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734fa36-b85d-4d06-975b-792045b28873 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.484696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60b519-f593-4f07-9f33-1c48519cbaf3 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.487433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734fa36-b85d-4d06-975b-792045b28873 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.489339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8508b7b-3f23-4e5e-81cf-c87b05969095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.489993 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8508b7b-3f23-4e5e-81cf-c87b05969095 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.490280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8508b7b-3f23-4e5e-81cf-c87b05969095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.490560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8508b7b-3f23-4e5e-81cf-c87b05969095 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.491296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8508b7b-3f23-4e5e-81cf-c87b05969095 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.492604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891f6b1-af2b-434d-a428-6617e79a578d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.493224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8508b7b-3f23-4e5e-81cf-c87b05969095 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.493417 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891f6b1-af2b-434d-a428-6617e79a578d admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.494281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891f6b1-af2b-434d-a428-6617e79a578d admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.494565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891f6b1-af2b-434d-a428-6617e79a578d admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.495250 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891f6b1-af2b-434d-a428-6617e79a578d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.497219 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891f6b1-af2b-434d-a428-6617e79a578d admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.514732 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 791/1591] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:38 2022] POST /identity/v3/users => generated 303 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:38.515582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8508b7b-3f23-4e5e-81cf-c87b05969095 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.522264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 801/1592] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:38 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:38.523307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891f6b1-af2b-434d-a428-6617e79a578d admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.530688 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8508b7b-3f23-4e5e-81cf-c87b05969095 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.533128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891f6b1-af2b-434d-a428-6617e79a578d admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.535441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138edec7-71da-4be7-9f3e-2aa416b069f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.536103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138edec7-71da-4be7-9f3e-2aa416b069f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.536387 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138edec7-71da-4be7-9f3e-2aa416b069f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.536662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138edec7-71da-4be7-9f3e-2aa416b069f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.537434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138edec7-71da-4be7-9f3e-2aa416b069f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.537737 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32306ed3-c974-4fea-a51a-21b77ae53e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.538379 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32306ed3-c974-4fea-a51a-21b77ae53e84 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.538665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32306ed3-c974-4fea-a51a-21b77ae53e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.538944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32306ed3-c974-4fea-a51a-21b77ae53e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.539151 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138edec7-71da-4be7-9f3e-2aa416b069f5 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.539663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32306ed3-c974-4fea-a51a-21b77ae53e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.541412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32306ed3-c974-4fea-a51a-21b77ae53e84 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.551031 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 792/1593] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:38.552063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 802/1594] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:38.552191 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138edec7-71da-4be7-9f3e-2aa416b069f5 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.554079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466de66b-7412-46b9-8f74-6b57d25d5e1e None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.554655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466de66b-7412-46b9-8f74-6b57d25d5e1e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.555160 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466de66b-7412-46b9-8f74-6b57d25d5e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.564306 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138edec7-71da-4be7-9f3e-2aa416b069f5 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.569706 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c06a5-9487-4860-b10c-9103aeda9bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.570375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c06a5-9487-4860-b10c-9103aeda9bae admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.570679 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c06a5-9487-4860-b10c-9103aeda9bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.570959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c06a5-9487-4860-b10c-9103aeda9bae admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.571718 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c06a5-9487-4860-b10c-9103aeda9bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.574004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c06a5-9487-4860-b10c-9103aeda9bae admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.582585 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-466de66b-7412-46b9-8f74-6b57d25d5e1e None None] MFA Rules not processed for user `e7d835605f074ba885baffa7d023f943`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:38.583804 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-466de66b-7412-46b9-8f74-6b57d25d5e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:38.584239 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 793/1595] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:38.585141 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c06a5-9487-4860-b10c-9103aeda9bae admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.596831 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c06a5-9487-4860-b10c-9103aeda9bae admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.601648 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdddf1-d862-4058-8e1a-28b67c123f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.602343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdddf1-d862-4058-8e1a-28b67c123f49 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.602629 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdddf1-d862-4058-8e1a-28b67c123f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.602909 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdddf1-d862-4058-8e1a-28b67c123f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.604342 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bdddf1-d862-4058-8e1a-28b67c123f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.605987 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bdddf1-d862-4058-8e1a-28b67c123f49 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.615789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 794/1596] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:38.616592 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdddf1-d862-4058-8e1a-28b67c123f49 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.626130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdddf1-d862-4058-8e1a-28b67c123f49 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.630708 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437a792-d92a-4a30-98cc-262e2509bf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.631365 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437a792-d92a-4a30-98cc-262e2509bf3d admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.631654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437a792-d92a-4a30-98cc-262e2509bf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.631936 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437a792-d92a-4a30-98cc-262e2509bf3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.632654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437a792-d92a-4a30-98cc-262e2509bf3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.634382 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437a792-d92a-4a30-98cc-262e2509bf3d admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.644881 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 795/1597] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:38.646137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437a792-d92a-4a30-98cc-262e2509bf3d admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.654906 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 803/1598] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:38 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:38.655831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466de66b-7412-46b9-8f74-6b57d25d5e1e None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.656451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437a792-d92a-4a30-98cc-262e2509bf3d admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.666029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4572f6-2874-4921-90f3-2a8a20895731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.667003 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4572f6-2874-4921-90f3-2a8a20895731 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.667299 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4572f6-2874-4921-90f3-2a8a20895731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.667584 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4572f6-2874-4921-90f3-2a8a20895731 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.667773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466de66b-7412-46b9-8f74-6b57d25d5e1e None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.668321 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4572f6-2874-4921-90f3-2a8a20895731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.670032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4572f6-2874-4921-90f3-2a8a20895731 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.675319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa50be6-0b00-4c32-b281-e1af26dac6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.676053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa50be6-0b00-4c32-b281-e1af26dac6db admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.676270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa50be6-0b00-4c32-b281-e1af26dac6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.676688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa50be6-0b00-4c32-b281-e1af26dac6db admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.677491 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa50be6-0b00-4c32-b281-e1af26dac6db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.679097 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa50be6-0b00-4c32-b281-e1af26dac6db admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.684271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 796/1599] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:38.684271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4572f6-2874-4921-90f3-2a8a20895731 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.693477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 804/1600] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:38.693477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa50be6-0b00-4c32-b281-e1af26dac6db admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.693713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4572f6-2874-4921-90f3-2a8a20895731 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.697126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced62f6c-312d-4024-ac24-a43a2fbabd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.697762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced62f6c-312d-4024-ac24-a43a2fbabd98 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.698088 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced62f6c-312d-4024-ac24-a43a2fbabd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.698368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced62f6c-312d-4024-ac24-a43a2fbabd98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.699103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced62f6c-312d-4024-ac24-a43a2fbabd98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.701123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced62f6c-312d-4024-ac24-a43a2fbabd98 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.702533 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa50be6-0b00-4c32-b281-e1af26dac6db admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.707935 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d8e6b-19e3-4579-9f97-be114622a15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.708883 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d8e6b-19e3-4579-9f97-be114622a15c admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.709171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d8e6b-19e3-4579-9f97-be114622a15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.709462 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d8e6b-19e3-4579-9f97-be114622a15c admin admin] PATH_INFO: `/v3/projects/453ee4bddca64aa5b4fd56447be26046/users/41b6585328bc4134a67820b5114c19c3/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.711405 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d8e6b-19e3-4579-9f97-be114622a15c admin admin] RBAC: Authorizing `identity:create_grant(project_id=453ee4bddca64aa5b4fd56447be26046, user_id=41b6585328bc4134a67820b5114c19c3, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.712252 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 797/1601] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:38.715053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d8e6b-19e3-4579-9f97-be114622a15c admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.716763 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced62f6c-312d-4024-ac24-a43a2fbabd98 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.725987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 805/1602] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:38 2022] PUT /identity/v3/projects/453ee4bddca64aa5b4fd56447be26046/users/41b6585328bc4134a67820b5114c19c3/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:38.726625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced62f6c-312d-4024-ac24-a43a2fbabd98 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.727038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d8e6b-19e3-4579-9f97-be114622a15c admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.792074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51315d7-d659-49d1-89e9-2a7ef1a083cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.793341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51315d7-d659-49d1-89e9-2a7ef1a083cb admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.793632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51315d7-d659-49d1-89e9-2a7ef1a083cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.793913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51315d7-d659-49d1-89e9-2a7ef1a083cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.794775 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51315d7-d659-49d1-89e9-2a7ef1a083cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.796330 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d8e6b-19e3-4579-9f97-be114622a15c admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.796608 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51315d7-d659-49d1-89e9-2a7ef1a083cb admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.801213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda04e6-22e7-476c-923c-00067a0791c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.802104 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda04e6-22e7-476c-923c-00067a0791c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.802386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda04e6-22e7-476c-923c-00067a0791c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.802662 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda04e6-22e7-476c-923c-00067a0791c6 admin admin] PATH_INFO: `/v3/projects/bdc783af528d4cd2a2b05519449924d9/users/bf13eeff770b406796ab7008b8429637/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.810901 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 798/1603] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:38.810901 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51315d7-d659-49d1-89e9-2a7ef1a083cb admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.818637 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda04e6-22e7-476c-923c-00067a0791c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc783af528d4cd2a2b05519449924d9, user_id=bf13eeff770b406796ab7008b8429637, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.820942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51315d7-d659-49d1-89e9-2a7ef1a083cb admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.822502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda04e6-22e7-476c-923c-00067a0791c6 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.826716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412692b5-f20e-47f2-89ac-3db0399d5be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.827891 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412692b5-f20e-47f2-89ac-3db0399d5be9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.828238 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412692b5-f20e-47f2-89ac-3db0399d5be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.828593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412692b5-f20e-47f2-89ac-3db0399d5be9 admin admin] PATH_INFO: `/v3/projects/1e4d6faeb79c4834a4ca61193f0cc657/users/85dea53a4a034f5981b638bf92f2cf3b/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.837038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 806/1604] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:38 2022] PUT /identity/v3/projects/bdc783af528d4cd2a2b05519449924d9/users/bf13eeff770b406796ab7008b8429637/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:38.841879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda04e6-22e7-476c-923c-00067a0791c6 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.845257 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412692b5-f20e-47f2-89ac-3db0399d5be9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4d6faeb79c4834a4ca61193f0cc657, user_id=85dea53a4a034f5981b638bf92f2cf3b, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.847950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412692b5-f20e-47f2-89ac-3db0399d5be9 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.852314 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:38.863496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 799/1605] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:38 2022] PUT /identity/v3/projects/1e4d6faeb79c4834a4ca61193f0cc657/users/85dea53a4a034f5981b638bf92f2cf3b/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:38.864368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412692b5-f20e-47f2-89ac-3db0399d5be9 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.871088 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:38.902752 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda04e6-22e7-476c-923c-00067a0791c6 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.937648 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412692b5-f20e-47f2-89ac-3db0399d5be9 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.942478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee87692-2570-4f53-b57d-85deb49f4322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.943150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee87692-2570-4f53-b57d-85deb49f4322 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.943431 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee87692-2570-4f53-b57d-85deb49f4322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.945173 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee87692-2570-4f53-b57d-85deb49f4322 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.945173 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee87692-2570-4f53-b57d-85deb49f4322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.946387 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee87692-2570-4f53-b57d-85deb49f4322 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.955662 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713314ab-9353-4298-b7da-6ede2ad5b706 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.956268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713314ab-9353-4298-b7da-6ede2ad5b706 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.956577 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713314ab-9353-4298-b7da-6ede2ad5b706 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.957083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 800/1606] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:38.957412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713314ab-9353-4298-b7da-6ede2ad5b706 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.957935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee87692-2570-4f53-b57d-85deb49f4322 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:38.973259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee87692-2570-4f53-b57d-85deb49f4322 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:38.978278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a91c1-bab0-4862-ab50-de150f74b50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:38.978978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a91c1-bab0-4862-ab50-de150f74b50a admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:38.979288 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a91c1-bab0-4862-ab50-de150f74b50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:38.979698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a91c1-bab0-4862-ab50-de150f74b50a admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:38.980475 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a91c1-bab0-4862-ab50-de150f74b50a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:38.982162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a91c1-bab0-4862-ab50-de150f74b50a admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:38.992587 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 801/1607] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:38.993416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a91c1-bab0-4862-ab50-de150f74b50a admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:39.005281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a91c1-bab0-4862-ab50-de150f74b50a admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:39.010416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61466bf-1dea-4360-92f0-d4113752d815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:39.011287 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61466bf-1dea-4360-92f0-d4113752d815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:39.011569 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61466bf-1dea-4360-92f0-d4113752d815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:39.011872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61466bf-1dea-4360-92f0-d4113752d815 admin admin] PATH_INFO: `/v3/projects/f7d10482f6bb42ca9e7af85a7478e97c/users/faa994a326a74892824eec2d26856312/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:39.020334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713314ab-9353-4298-b7da-6ede2ad5b706 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:39.021814 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713314ab-9353-4298-b7da-6ede2ad5b706 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:39.027183 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61466bf-1dea-4360-92f0-d4113752d815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d10482f6bb42ca9e7af85a7478e97c, user_id=faa994a326a74892824eec2d26856312, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:39.030019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61466bf-1dea-4360-92f0-d4113752d815 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:39.031233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 807/1608] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:39.032637 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f07d42-9691-46e8-9e40-6287f5d84f72 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:39.032960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f07d42-9691-46e8-9e40-6287f5d84f72 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:39.033238 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f07d42-9691-46e8-9e40-6287f5d84f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:39.071951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 802/1609] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:38 2022] PUT /identity/v3/projects/f7d10482f6bb42ca9e7af85a7478e97c/users/faa994a326a74892824eec2d26856312/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:39.071951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61466bf-1dea-4360-92f0-d4113752d815 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:39.072448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-15f07d42-9691-46e8-9e40-6287f5d84f72 None None] MFA Rules not processed for user `41b6585328bc4134a67820b5114c19c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:39.072448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-15f07d42-9691-46e8-9e40-6287f5d84f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:39.143750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61466bf-1dea-4360-92f0-d4113752d815 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:39.149419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c59a00-3dc2-4797-8497-72a13cd5296d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:39.150350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c59a00-3dc2-4797-8497-72a13cd5296d admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:39.150640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c59a00-3dc2-4797-8497-72a13cd5296d admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:39.150923 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c59a00-3dc2-4797-8497-72a13cd5296d admin admin] PATH_INFO: `/v3/projects/807d2d1b808448bca8aaf30389c8c571/users/4a746b42824c4d13a1d0b0f2c5831259/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:39.153684 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 808/1610] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:39 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:39.155184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e687d16-73fc-4fb5-b8af-56516eb47e98 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:39.155499 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e687d16-73fc-4fb5-b8af-56516eb47e98 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:39.156104 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e687d16-73fc-4fb5-b8af-56516eb47e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:39.168404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:39.173770 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c59a00-3dc2-4797-8497-72a13cd5296d admin admin] RBAC: Authorizing `identity:create_grant(project_id=807d2d1b808448bca8aaf30389c8c571, user_id=4a746b42824c4d13a1d0b0f2c5831259, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:39.177413 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c59a00-3dc2-4797-8497-72a13cd5296d admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:39.185286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-1e687d16-73fc-4fb5-b8af-56516eb47e98 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:39.186583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-1e687d16-73fc-4fb5-b8af-56516eb47e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:39.195148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 803/1611] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:39 2022] PUT /identity/v3/projects/807d2d1b808448bca8aaf30389c8c571/users/4a746b42824c4d13a1d0b0f2c5831259/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:39.196790 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f8b37-4ad9-4c05-988b-40b5ec2f2ca0 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:39.197174 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f8b37-4ad9-4c05-988b-40b5ec2f2ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:39.197530 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f8b37-4ad9-4c05-988b-40b5ec2f2ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:39.216456 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-883f8b37-4ad9-4c05-988b-40b5ec2f2ca0 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:39.217705 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-883f8b37-4ad9-4c05-988b-40b5ec2f2ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:39.258513 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 809/1612] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:39.259496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e687d16-73fc-4fb5-b8af-56516eb47e98 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:39.268344 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:39.271920 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e687d16-73fc-4fb5-b8af-56516eb47e98 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:39.281361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd7e28-8a0b-49f7-a170-cbf080850398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:39.282391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd7e28-8a0b-49f7-a170-cbf080850398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:39.282773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd7e28-8a0b-49f7-a170-cbf080850398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:39.283168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd7e28-8a0b-49f7-a170-cbf080850398 admin admin] PATH_INFO: `/v3/projects/2bfddae441b9434eb18589926f360876/users/21a02a34ae414ea3a3afbefe29709b74/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:39.297039 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 804/1613] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:39.298136 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f8b37-4ad9-4c05-988b-40b5ec2f2ca0 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:39.302854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd7e28-8a0b-49f7-a170-cbf080850398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bfddae441b9434eb18589926f360876, user_id=21a02a34ae414ea3a3afbefe29709b74, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:39.305692 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd7e28-8a0b-49f7-a170-cbf080850398 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:39.310066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f8b37-4ad9-4c05-988b-40b5ec2f2ca0 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:39.316117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf64991-591e-4bea-909b-9d78c2d3e97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:39.317143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf64991-591e-4bea-909b-9d78c2d3e97c admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:39.317437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf64991-591e-4bea-909b-9d78c2d3e97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:39.317701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf64991-591e-4bea-909b-9d78c2d3e97c admin admin] PATH_INFO: `/v3/projects/c9496181d0314661a8645a5fc6f6f8a2/users/7026b47ce1074361938e69abfd18f5fd/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:39.324216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 810/1614] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:39 2022] PUT /identity/v3/projects/2bfddae441b9434eb18589926f360876/users/21a02a34ae414ea3a3afbefe29709b74/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:39.326070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b5256-e764-41bb-accc-08c43f712e31 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:39.326366 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b5256-e764-41bb-accc-08c43f712e31 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:39.326655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b5256-e764-41bb-accc-08c43f712e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:39.341120 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf64991-591e-4bea-909b-9d78c2d3e97c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9496181d0314661a8645a5fc6f6f8a2, user_id=7026b47ce1074361938e69abfd18f5fd, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:39.344157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf64991-591e-4bea-909b-9d78c2d3e97c admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:39.358850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 805/1615] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:39 2022] PUT /identity/v3/projects/c9496181d0314661a8645a5fc6f6f8a2/users/7026b47ce1074361938e69abfd18f5fd/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:39.359707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf64991-591e-4bea-909b-9d78c2d3e97c admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:39.375144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-876b5256-e764-41bb-accc-08c43f712e31 None None] MFA Rules not processed for user `faa994a326a74892824eec2d26856312`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:39.376469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-876b5256-e764-41bb-accc-08c43f712e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:39.428880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf64991-591e-4bea-909b-9d78c2d3e97c admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:39.433339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a05a5c-cb5e-4d4c-8393-6e41bb0f2deb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:39.434086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a05a5c-cb5e-4d4c-8393-6e41bb0f2deb service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:39.434449 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a05a5c-cb5e-4d4c-8393-6e41bb0f2deb service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:39.434800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a05a5c-cb5e-4d4c-8393-6e41bb0f2deb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:39.447415 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 811/1616] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:39 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:39.448951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc4b07-01bd-411a-8b8a-da98c120d145 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:39.449249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc4b07-01bd-411a-8b8a-da98c120d145 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:39.449537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc4b07-01bd-411a-8b8a-da98c120d145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:39.470477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-1dcc4b07-01bd-411a-8b8a-da98c120d145 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:39.471775 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-1dcc4b07-01bd-411a-8b8a-da98c120d145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:39.514661 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a05a5c-cb5e-4d4c-8393-6e41bb0f2deb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:39.521164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a05a5c-cb5e-4d4c-8393-6e41bb0f2deb service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:39.542209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 812/1617] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:39.543221 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc4b07-01bd-411a-8b8a-da98c120d145 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:39.552686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 806/1618] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:39 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:39.554516 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a05a5c-cb5e-4d4c-8393-6e41bb0f2deb service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:39.581194 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:39.652965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc4b07-01bd-411a-8b8a-da98c120d145 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:39.655996 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb71763-4831-4d3e-b32b-0c0e9c0397c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:39.656937 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb71763-4831-4d3e-b32b-0c0e9c0397c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:39.657341 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb71763-4831-4d3e-b32b-0c0e9c0397c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:39.657795 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb71763-4831-4d3e-b32b-0c0e9c0397c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:39.668232 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a05a5c-cb5e-4d4c-8393-6e41bb0f2deb service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:39.672115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef81b3f-e7ce-4fba-b3af-254847aa626c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:39.676893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb71763-4831-4d3e-b32b-0c0e9c0397c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:39.677651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef81b3f-e7ce-4fba-b3af-254847aa626c service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:39.678016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef81b3f-e7ce-4fba-b3af-254847aa626c service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:39.678419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef81b3f-e7ce-4fba-b3af-254847aa626c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:39.679414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb71763-4831-4d3e-b32b-0c0e9c0397c2 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:39.689784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef81b3f-e7ce-4fba-b3af-254847aa626c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:39.691246 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef81b3f-e7ce-4fba-b3af-254847aa626c service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:39.691776 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 813/1619] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:39.698857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf81126-5259-46b2-881d-925c42191f3d None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:39.699152 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf81126-5259-46b2-881d-925c42191f3d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:39.699431 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf81126-5259-46b2-881d-925c42191f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:39.707885 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 807/1620] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:39.709134 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda84c7c-d7d9-4a9a-99f0-151a73dd1093 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:39.709427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda84c7c-d7d9-4a9a-99f0-151a73dd1093 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:39.709705 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda84c7c-d7d9-4a9a-99f0-151a73dd1093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:39.726378 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-6cf81126-5259-46b2-881d-925c42191f3d None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:39.729634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-6cf81126-5259-46b2-881d-925c42191f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:39.749022 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 814/1621] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:39.750680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf81126-5259-46b2-881d-925c42191f3d None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:39.751806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-fda84c7c-d7d9-4a9a-99f0-151a73dd1093 None None] MFA Rules not processed for user `7026b47ce1074361938e69abfd18f5fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:39.753536 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-fda84c7c-d7d9-4a9a-99f0-151a73dd1093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:39.758978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:39.765961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf81126-5259-46b2-881d-925c42191f3d None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:39.774160 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5b99a-f566-48a7-91de-9c7c4da66f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:39.775078 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5b99a-f566-48a7-91de-9c7c4da66f90 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:39.775588 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5b99a-f566-48a7-91de-9c7c4da66f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:39.776190 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5b99a-f566-48a7-91de-9c7c4da66f90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:39.776989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5b99a-f566-48a7-91de-9c7c4da66f90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:39.779152 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5b99a-f566-48a7-91de-9c7c4da66f90 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:39.818823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 815/1622] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:39 2022] POST /identity/v3/projects => generated 369 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:39.820015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5b99a-f566-48a7-91de-9c7c4da66f90 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:39.845244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 808/1623] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:39.846466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda84c7c-d7d9-4a9a-99f0-151a73dd1093 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:39.862762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda84c7c-d7d9-4a9a-99f0-151a73dd1093 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:39.910799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5b99a-f566-48a7-91de-9c7c4da66f90 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:39.914930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43900c-d9d6-42af-bc4e-8d88b75fda61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:39.916224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43900c-d9d6-42af-bc4e-8d88b75fda61 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:39.916499 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43900c-d9d6-42af-bc4e-8d88b75fda61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:39.916778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43900c-d9d6-42af-bc4e-8d88b75fda61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:39.942849 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970bff6-6641-4aec-95ab-37a8ab9d9ac8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:39.943541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970bff6-6641-4aec-95ab-37a8ab9d9ac8 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:39.944051 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970bff6-6641-4aec-95ab-37a8ab9d9ac8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:39.944217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970bff6-6641-4aec-95ab-37a8ab9d9ac8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:39.996754 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43900c-d9d6-42af-bc4e-8d88b75fda61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:39.998544 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43900c-d9d6-42af-bc4e-8d88b75fda61 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:40.017657 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 816/1624] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:40.018594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43900c-d9d6-42af-bc4e-8d88b75fda61 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:40.024193 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970bff6-6641-4aec-95ab-37a8ab9d9ac8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:40.026152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970bff6-6641-4aec-95ab-37a8ab9d9ac8 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:40.027724 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43900c-d9d6-42af-bc4e-8d88b75fda61 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:40.032058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65cbc4-95c6-43ec-8d72-c510cd7d9473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:40.032814 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65cbc4-95c6-43ec-8d72-c510cd7d9473 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:40.033176 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65cbc4-95c6-43ec-8d72-c510cd7d9473 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:40.033531 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65cbc4-95c6-43ec-8d72-c510cd7d9473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:40.036635 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 809/1625] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:39 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:40.051133 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65cbc4-95c6-43ec-8d72-c510cd7d9473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:40.054579 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65cbc4-95c6-43ec-8d72-c510cd7d9473 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:40.056535 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970bff6-6641-4aec-95ab-37a8ab9d9ac8 service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:40.066878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 817/1626] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:40.068845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d32b37-64ae-48bb-97d3-7b3f4751fec0 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:40.069714 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d32b37-64ae-48bb-97d3-7b3f4751fec0 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:40.070237 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d32b37-64ae-48bb-97d3-7b3f4751fec0 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:40.071880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970bff6-6641-4aec-95ab-37a8ab9d9ac8 service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:40.072735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 818/1627] 10.222.0.44 () {54 vars in 961 bytes} [Tue Jul 12 22:05:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:05:40.077032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728ff9f-b93e-400e-88bf-ed20c89e0168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:40.077713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728ff9f-b93e-400e-88bf-ed20c89e0168 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:40.077991 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728ff9f-b93e-400e-88bf-ed20c89e0168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:40.078269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728ff9f-b93e-400e-88bf-ed20c89e0168 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:40.078958 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728ff9f-b93e-400e-88bf-ed20c89e0168 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:40.080457 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae855260-1ebd-4217-b57d-0d541376a7dc None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:40.081214 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728ff9f-b93e-400e-88bf-ed20c89e0168 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:40.081619 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae855260-1ebd-4217-b57d-0d541376a7dc None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:40.082254 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae855260-1ebd-4217-b57d-0d541376a7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:40.117578 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 810/1628] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:40 2022] POST /identity/v3/users => generated 291 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:40.122414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728ff9f-b93e-400e-88bf-ed20c89e0168 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:40.136516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-ae855260-1ebd-4217-b57d-0d541376a7dc None None] MFA Rules not processed for user `24915625354f418eb7bc4d5ada85fdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:40.137813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ae855260-1ebd-4217-b57d-0d541376a7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:40.143862 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:40.150960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728ff9f-b93e-400e-88bf-ed20c89e0168 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:40.161252 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6ae79-bd39-4c6d-b7c5-9b9352ab12e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:40.161766 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6ae79-bd39-4c6d-b7c5-9b9352ab12e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:40.162187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6ae79-bd39-4c6d-b7c5-9b9352ab12e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:40.162473 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6ae79-bd39-4c6d-b7c5-9b9352ab12e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:40.163447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b6ae79-bd39-4c6d-b7c5-9b9352ab12e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:40.170466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b6ae79-bd39-4c6d-b7c5-9b9352ab12e9 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:40.196211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 811/1629] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:05:40.197898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6ae79-bd39-4c6d-b7c5-9b9352ab12e9 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:40.221166 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6ae79-bd39-4c6d-b7c5-9b9352ab12e9 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:40.232255 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab1f0c-99a8-4290-9d42-eae9a8aa125b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:40.233274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab1f0c-99a8-4290-9d42-eae9a8aa125b admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:40.233631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab1f0c-99a8-4290-9d42-eae9a8aa125b admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:40.233979 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab1f0c-99a8-4290-9d42-eae9a8aa125b admin admin] PATH_INFO: `/v3/projects/f98dc7df181e45d98c1b23e061d556a3/users/8dfeeaffaf884f95bb0f8f30583a8fd3/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:40.242300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 819/1630] 10.222.0.44 () {58 vars in 1058 bytes} [Tue Jul 12 22:05:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:40.253824 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae855260-1ebd-4217-b57d-0d541376a7dc None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:40.259332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab1f0c-99a8-4290-9d42-eae9a8aa125b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f98dc7df181e45d98c1b23e061d556a3, user_id=8dfeeaffaf884f95bb0f8f30583a8fd3, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:40.262270 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab1f0c-99a8-4290-9d42-eae9a8aa125b admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:40.266171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae855260-1ebd-4217-b57d-0d541376a7dc None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:40.273794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575d442-5203-45c7-9415-6b75ad8a44cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:40.274628 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575d442-5203-45c7-9415-6b75ad8a44cd service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:40.275724 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575d442-5203-45c7-9415-6b75ad8a44cd service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:40.276072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575d442-5203-45c7-9415-6b75ad8a44cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:40.281260 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 812/1631] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:40 2022] PUT /identity/v3/projects/f98dc7df181e45d98c1b23e061d556a3/users/8dfeeaffaf884f95bb0f8f30583a8fd3/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:05:40.288865 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab1f0c-99a8-4290-9d42-eae9a8aa125b admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:40.312256 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:40.353638 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575d442-5203-45c7-9415-6b75ad8a44cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:40.355774 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575d442-5203-45c7-9415-6b75ad8a44cd service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:40.368224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 820/1632] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:05:40.394169 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab1f0c-99a8-4290-9d42-eae9a8aa125b admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:40.400979 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f4cb6-ca1e-4eb1-a4ce-2b2a179dd86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:40.401844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f4cb6-ca1e-4eb1-a4ce-2b2a179dd86e admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:40.403137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f4cb6-ca1e-4eb1-a4ce-2b2a179dd86e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:40.403137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f4cb6-ca1e-4eb1-a4ce-2b2a179dd86e admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:40.403592 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0f4cb6-ca1e-4eb1-a4ce-2b2a179dd86e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:40.405365 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0f4cb6-ca1e-4eb1-a4ce-2b2a179dd86e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:40.428171 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 813/1633] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:40.429672 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575d442-5203-45c7-9415-6b75ad8a44cd service nova] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:40.444085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575d442-5203-45c7-9415-6b75ad8a44cd service nova] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:40.451977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed5699-c5a6-47e9-8a1d-60b1bccb0d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:40.453628 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed5699-c5a6-47e9-8a1d-60b1bccb0d22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:40.453918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed5699-c5a6-47e9-8a1d-60b1bccb0d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:40.454200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed5699-c5a6-47e9-8a1d-60b1bccb0d22 admin admin] PATH_INFO: `/v3/projects/f98dc7df181e45d98c1b23e061d556a3/users/8dfeeaffaf884f95bb0f8f30583a8fd3/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:40.456390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed5699-c5a6-47e9-8a1d-60b1bccb0d22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f98dc7df181e45d98c1b23e061d556a3, user_id=8dfeeaffaf884f95bb0f8f30583a8fd3, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:40.460119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed5699-c5a6-47e9-8a1d-60b1bccb0d22 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:40.476750 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 821/1634] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:40 2022] PUT /identity/v3/projects/f98dc7df181e45d98c1b23e061d556a3/users/8dfeeaffaf884f95bb0f8f30583a8fd3/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:40.485547 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536809c1-8496-42c5-855f-9060795e8286 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:40.485952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536809c1-8496-42c5-855f-9060795e8286 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:40.486323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536809c1-8496-42c5-855f-9060795e8286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:40.515338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-536809c1-8496-42c5-855f-9060795e8286 None None] MFA Rules not processed for user `8dfeeaffaf884f95bb0f8f30583a8fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:40.517057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-536809c1-8496-42c5-855f-9060795e8286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:40.635022 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811236-bc40-4d24-b5b9-d76f36e0ee8e None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:40.635778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811236-bc40-4d24-b5b9-d76f36e0ee8e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:40.635909 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811236-bc40-4d24-b5b9-d76f36e0ee8e None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:40.638112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 822/1635] 10.222.0.44 () {54 vars in 912 bytes} [Tue Jul 12 22:05:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 22:05:40.644454 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3eb617-b5ff-4eff-80a6-666f8b1bc6aa None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:40.644454 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3eb617-b5ff-4eff-80a6-666f8b1bc6aa None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:40.644866 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3eb617-b5ff-4eff-80a6-666f8b1bc6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:40.667015 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 814/1636] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:40 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:40.668003 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-7e3eb617-b5ff-4eff-80a6-666f8b1bc6aa None None] MFA Rules not processed for user `24915625354f418eb7bc4d5ada85fdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:40.668580 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536809c1-8496-42c5-855f-9060795e8286 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:40.670248 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-7e3eb617-b5ff-4eff-80a6-666f8b1bc6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:40.770507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 823/1637] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Jul 12 22:05:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:40.800527 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536809c1-8496-42c5-855f-9060795e8286 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:40.804482 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57732a-0dbd-4699-8d80-89471665cb39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:40.805366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57732a-0dbd-4699-8d80-89471665cb39 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:40.805926 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57732a-0dbd-4699-8d80-89471665cb39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:40.806081 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57732a-0dbd-4699-8d80-89471665cb39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:40.817246 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3eb617-b5ff-4eff-80a6-666f8b1bc6aa None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:40.817528 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57732a-0dbd-4699-8d80-89471665cb39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:40.829587 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3eb617-b5ff-4eff-80a6-666f8b1bc6aa None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:40.831612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57732a-0dbd-4699-8d80-89471665cb39 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:40.840480 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901a42b-e9bc-4e78-a3da-8aee1c82705e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:40.841286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901a42b-e9bc-4e78-a3da-8aee1c82705e service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:40.841652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901a42b-e9bc-4e78-a3da-8aee1c82705e service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:40.842027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901a42b-e9bc-4e78-a3da-8aee1c82705e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:40.867207 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901a42b-e9bc-4e78-a3da-8aee1c82705e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:40.870444 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901a42b-e9bc-4e78-a3da-8aee1c82705e service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:40.883292 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 815/1638] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:40 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:40.910762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 824/1639] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:40.965901 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57732a-0dbd-4699-8d80-89471665cb39 service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:40.983060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57732a-0dbd-4699-8d80-89471665cb39 service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:40.986970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cca2f6-0977-4040-a725-f3c8c5cc7e14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:40.987631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cca2f6-0977-4040-a725-f3c8c5cc7e14 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:40.987914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cca2f6-0977-4040-a725-f3c8c5cc7e14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:40.988189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cca2f6-0977-4040-a725-f3c8c5cc7e14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:41.078620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:41.148184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380260f-933c-458e-889b-930d56355081 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:41.148481 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380260f-933c-458e-889b-930d56355081 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:41.148888 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380260f-933c-458e-889b-930d56355081 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:41.151365 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 825/1640] 10.222.0.44 () {54 vars in 961 bytes} [Tue Jul 12 22:05:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:05:41.166575 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16c942-d4ec-458d-a8b9-b8e011da0cef None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:41.166861 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16c942-d4ec-458d-a8b9-b8e011da0cef None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:41.167172 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16c942-d4ec-458d-a8b9-b8e011da0cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:41.178828 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:41.191334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cca2f6-0977-4040-a725-f3c8c5cc7e14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:41.194774 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-2d16c942-d4ec-458d-a8b9-b8e011da0cef None None] MFA Rules not processed for user `24915625354f418eb7bc4d5ada85fdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:41.195144 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cca2f6-0977-4040-a725-f3c8c5cc7e14 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:41.196490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2d16c942-d4ec-458d-a8b9-b8e011da0cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:41.214253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 826/1641] 10.222.0.44 () {58 vars in 1058 bytes} [Tue Jul 12 22:05:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:05:41.220393 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462db347-dd67-4008-9bd0-1c3703f3f1af None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:41.220753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462db347-dd67-4008-9bd0-1c3703f3f1af None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:41.233300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462db347-dd67-4008-9bd0-1c3703f3f1af None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:41.234698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 816/1642] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:40 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:41.237739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 827/1643] 10.222.0.44 () {54 vars in 912 bytes} [Tue Jul 12 22:05:41 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:05:41.242491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dc5e4-8d6c-438a-8d36-60713bfe794e None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:41.242711 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dc5e4-8d6c-438a-8d36-60713bfe794e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:41.242971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dc5e4-8d6c-438a-8d36-60713bfe794e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:41.285959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-6c3dc5e4-8d6c-438a-8d36-60713bfe794e None None] MFA Rules not processed for user `24915625354f418eb7bc4d5ada85fdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:41.287202 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6c3dc5e4-8d6c-438a-8d36-60713bfe794e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:41.307108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 817/1644] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Jul 12 22:05:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:41.636532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462db347-dd67-4008-9bd0-1c3703f3f1af None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:41.651795 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462db347-dd67-4008-9bd0-1c3703f3f1af None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:41.657647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05857b6-8379-4c97-b184-76258b27f794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:41.658771 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05857b6-8379-4c97-b184-76258b27f794 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:41.659197 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05857b6-8379-4c97-b184-76258b27f794 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:41.659700 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05857b6-8379-4c97-b184-76258b27f794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:41.672675 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05857b6-8379-4c97-b184-76258b27f794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:41.674785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05857b6-8379-4c97-b184-76258b27f794 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:41.706571 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 828/1645] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:42.644759 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6d554-4b16-4239-b822-23e4864d54b1 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:42.645350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6d554-4b16-4239-b822-23e4864d54b1 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:42.645622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6d554-4b16-4239-b822-23e4864d54b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:42.720647 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-80d6d554-4b16-4239-b822-23e4864d54b1 None None] MFA Rules not processed for user `21a02a34ae414ea3a3afbefe29709b74`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:42.722895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-80d6d554-4b16-4239-b822-23e4864d54b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:43.012911 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 818/1646] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:42 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:05:43.013199 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05857b6-8379-4c97-b184-76258b27f794 service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:43.174548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05857b6-8379-4c97-b184-76258b27f794 service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:43.194263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a26231-8af6-4a88-8910-c438168fc042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:43.198284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a26231-8af6-4a88-8910-c438168fc042 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:43.198576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a26231-8af6-4a88-8910-c438168fc042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:43.198867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a26231-8af6-4a88-8910-c438168fc042 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:43.199805 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a26231-8af6-4a88-8910-c438168fc042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:43.201904 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a26231-8af6-4a88-8910-c438168fc042 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:43.271364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6d554-4b16-4239-b822-23e4864d54b1 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:43.278054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 829/1647] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:43 2022] POST /identity/v3/projects => generated 403 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:43.289358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6d554-4b16-4239-b822-23e4864d54b1 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:43.469107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19b962-c29f-4408-b6d9-144e2ba18fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:43.470924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19b962-c29f-4408-b6d9-144e2ba18fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:43.471424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19b962-c29f-4408-b6d9-144e2ba18fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:43.472113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19b962-c29f-4408-b6d9-144e2ba18fb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:43.473080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19b962-c29f-4408-b6d9-144e2ba18fb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:43.476186 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19b962-c29f-4408-b6d9-144e2ba18fb1 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:43.532348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a26231-8af6-4a88-8910-c438168fc042 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:43.547782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a26231-8af6-4a88-8910-c438168fc042 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:43.548591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 819/1648] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:43 2022] POST /identity/v3/users => generated 308 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:05:43.565639 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d3260-0430-47d6-9e4c-0c3a03a23514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:43.567086 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d3260-0430-47d6-9e4c-0c3a03a23514 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:43.567653 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d3260-0430-47d6-9e4c-0c3a03a23514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:43.586448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d3260-0430-47d6-9e4c-0c3a03a23514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:43.588856 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d3260-0430-47d6-9e4c-0c3a03a23514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:43.595988 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d3260-0430-47d6-9e4c-0c3a03a23514 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:43.629275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 830/1649] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:05:43.632292 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19b962-c29f-4408-b6d9-144e2ba18fb1 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:43.647976 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19b962-c29f-4408-b6d9-144e2ba18fb1 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:43.656341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726024c1-2591-4056-97ff-e7251b8081cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:43.658039 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726024c1-2591-4056-97ff-e7251b8081cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:43.659187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726024c1-2591-4056-97ff-e7251b8081cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:43.659187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726024c1-2591-4056-97ff-e7251b8081cd admin admin] PATH_INFO: `/v3/projects/4dec303b5897499880c705ba996bd428/users/9cdeaf848c1f4fd8b110f2ceeadd5b62/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:43.680702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726024c1-2591-4056-97ff-e7251b8081cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dec303b5897499880c705ba996bd428, user_id=9cdeaf848c1f4fd8b110f2ceeadd5b62, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:43.684311 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726024c1-2591-4056-97ff-e7251b8081cd admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:43.706251 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 820/1650] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:43 2022] PUT /identity/v3/projects/4dec303b5897499880c705ba996bd428/users/9cdeaf848c1f4fd8b110f2ceeadd5b62/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:43.710321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d3260-0430-47d6-9e4c-0c3a03a23514 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:43.721773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d3260-0430-47d6-9e4c-0c3a03a23514 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:43.810338 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41102d4a-47ae-4544-8061-b35dfd9daa12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:43.811241 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41102d4a-47ae-4544-8061-b35dfd9daa12 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:43.811730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41102d4a-47ae-4544-8061-b35dfd9daa12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:43.812277 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41102d4a-47ae-4544-8061-b35dfd9daa12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:43.813286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41102d4a-47ae-4544-8061-b35dfd9daa12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:43.815207 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41102d4a-47ae-4544-8061-b35dfd9daa12 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:43.829673 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 831/1651] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:43.834214 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726024c1-2591-4056-97ff-e7251b8081cd admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:43.853445 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726024c1-2591-4056-97ff-e7251b8081cd admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:43.871067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7acbb-bcd9-40a2-9445-edd06175e66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:43.872762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7acbb-bcd9-40a2-9445-edd06175e66a admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:43.874859 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7acbb-bcd9-40a2-9445-edd06175e66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:43.875678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7acbb-bcd9-40a2-9445-edd06175e66a admin admin] PATH_INFO: `/v3/projects/4dec303b5897499880c705ba996bd428/users/9cdeaf848c1f4fd8b110f2ceeadd5b62/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:43.884450 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7acbb-bcd9-40a2-9445-edd06175e66a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dec303b5897499880c705ba996bd428, user_id=9cdeaf848c1f4fd8b110f2ceeadd5b62, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:43.889133 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7acbb-bcd9-40a2-9445-edd06175e66a admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:43.925283 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 821/1652] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:43 2022] PUT /identity/v3/projects/4dec303b5897499880c705ba996bd428/users/9cdeaf848c1f4fd8b110f2ceeadd5b62/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:44.771647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41102d4a-47ae-4544-8061-b35dfd9daa12 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:44.790915 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41102d4a-47ae-4544-8061-b35dfd9daa12 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:44.882165 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7acbb-bcd9-40a2-9445-edd06175e66a admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:44.891509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:44.901202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:44.921663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa86a07-4ff9-4d10-8768-d7634a3da2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:44.926941 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa86a07-4ff9-4d10-8768-d7634a3da2c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:44.928387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa86a07-4ff9-4d10-8768-d7634a3da2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:44.929420 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa86a07-4ff9-4d10-8768-d7634a3da2c2 admin admin] PATH_INFO: `/v3/users/518be0cad1d64ddabc1ab41ef3e456a6` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:44.932905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa86a07-4ff9-4d10-8768-d7634a3da2c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=518be0cad1d64ddabc1ab41ef3e456a6)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:44.941509 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa86a07-4ff9-4d10-8768-d7634a3da2c2 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:45.106024 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-0aa86a07-4ff9-4d10-8768-d7634a3da2c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '518be0cad1d64ddabc1ab41ef3e456a6'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:45.114052 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7acbb-bcd9-40a2-9445-edd06175e66a admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:45.127352 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-0aa86a07-4ff9-4d10-8768-d7634a3da2c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '518be0cad1d64ddabc1ab41ef3e456a6'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:45.128221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6144acf-01cc-4c5c-baae-0d6ea598739a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:45.130897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6144acf-01cc-4c5c-baae-0d6ea598739a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:45.131197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6144acf-01cc-4c5c-baae-0d6ea598739a admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:45.131478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6144acf-01cc-4c5c-baae-0d6ea598739a admin admin] PATH_INFO: `/v3/users/7026b47ce1074361938e69abfd18f5fd` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:45.135800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6144acf-01cc-4c5c-baae-0d6ea598739a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7026b47ce1074361938e69abfd18f5fd)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:45.137024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: WARNING py.warnings [None req-f6144acf-01cc-4c5c-baae-0d6ea598739a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 22:05:45.137024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: warnings.warn(msg) Jul 12 22:05:45.137024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: Jul 12 22:05:45.140408 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-0aa86a07-4ff9-4d10-8768-d7634a3da2c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '518be0cad1d64ddabc1ab41ef3e456a6'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:45.140565 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6144acf-01cc-4c5c-baae-0d6ea598739a admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:45.152420 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-0aa86a07-4ff9-4d10-8768-d7634a3da2c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '518be0cad1d64ddabc1ab41ef3e456a6'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:45.153560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-0aa86a07-4ff9-4d10-8768-d7634a3da2c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '518be0cad1d64ddabc1ab41ef3e456a6'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:45.174154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 832/1653] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:05:44 2022] DELETE /identity/v3/users/518be0cad1d64ddabc1ab41ef3e456a6 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:45.178130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb3bd-abf3-4707-bc07-c38aa94f34e9 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:45.178810 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb3bd-abf3-4707-bc07-c38aa94f34e9 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:45.179608 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffb3bd-abf3-4707-bc07-c38aa94f34e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:45.223725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-32ffb3bd-abf3-4707-bc07-c38aa94f34e9 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:45.225284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-32ffb3bd-abf3-4707-bc07-c38aa94f34e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:45.326971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f6144acf-01cc-4c5c-baae-0d6ea598739a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7026b47ce1074361938e69abfd18f5fd'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:45.377877 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f6144acf-01cc-4c5c-baae-0d6ea598739a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7026b47ce1074361938e69abfd18f5fd'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:45.414744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f6144acf-01cc-4c5c-baae-0d6ea598739a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7026b47ce1074361938e69abfd18f5fd'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:45.415681 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f6144acf-01cc-4c5c-baae-0d6ea598739a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7026b47ce1074361938e69abfd18f5fd'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:45.416150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f6144acf-01cc-4c5c-baae-0d6ea598739a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7026b47ce1074361938e69abfd18f5fd'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:45.445555 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 833/1654] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:45.482019 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffb3bd-abf3-4707-bc07-c38aa94f34e9 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:45.486405 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-32ffb3bd-abf3-4707-bc07-c38aa94f34e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:45.512167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 822/1655] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:05:44 2022] DELETE /identity/v3/users/7026b47ce1074361938e69abfd18f5fd => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:05:45.517958 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ce813-f9e0-4050-8e75-b0b008f7b245 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:45.524368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ce813-f9e0-4050-8e75-b0b008f7b245 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:45.525975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ce813-f9e0-4050-8e75-b0b008f7b245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:45.594124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-e03ce813-f9e0-4050-8e75-b0b008f7b245 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:45.595613 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e03ce813-f9e0-4050-8e75-b0b008f7b245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:45.701445 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffb3bd-abf3-4707-bc07-c38aa94f34e9 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:45.798558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:45.866449 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 823/1656] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:45.885860 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ce813-f9e0-4050-8e75-b0b008f7b245 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:45.895449 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b9c3e-eaa2-40d2-8e7c-437b0a9ec839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:45.901942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b9c3e-eaa2-40d2-8e7c-437b0a9ec839 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:45.902236 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b9c3e-eaa2-40d2-8e7c-437b0a9ec839 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:45.902523 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b9c3e-eaa2-40d2-8e7c-437b0a9ec839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:45.909776 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ce813-f9e0-4050-8e75-b0b008f7b245 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:45.924043 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cae2a-dd52-4efa-869d-ac92c78b1f32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:45.929422 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cae2a-dd52-4efa-869d-ac92c78b1f32 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:45.929991 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cae2a-dd52-4efa-869d-ac92c78b1f32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:45.930257 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cae2a-dd52-4efa-869d-ac92c78b1f32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:45.945256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b9c3e-eaa2-40d2-8e7c-437b0a9ec839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:45.949528 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b9c3e-eaa2-40d2-8e7c-437b0a9ec839 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:45.973597 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5cae2a-dd52-4efa-869d-ac92c78b1f32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:45.975731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5cae2a-dd52-4efa-869d-ac92c78b1f32 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:45.983674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 834/1657] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:45.996204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 824/1658] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:05:47.157794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b9c3e-eaa2-40d2-8e7c-437b0a9ec839 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:47.159409 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b67b9c3e-eaa2-40d2-8e7c-437b0a9ec839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:47.190448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b9c3e-eaa2-40d2-8e7c-437b0a9ec839 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:47.208058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d72e65-0313-498c-bf02-7633549fc79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:47.209032 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d72e65-0313-498c-bf02-7633549fc79c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:47.209319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d72e65-0313-498c-bf02-7633549fc79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:47.209934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d72e65-0313-498c-bf02-7633549fc79c admin admin] PATH_INFO: `/v3/users/faa994a326a74892824eec2d26856312` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:47.212866 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d72e65-0313-498c-bf02-7633549fc79c admin admin] RBAC: Authorizing `identity:delete_user(user_id=faa994a326a74892824eec2d26856312)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:47.216506 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d72e65-0313-498c-bf02-7633549fc79c admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:47.339904 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-b4d72e65-0313-498c-bf02-7633549fc79c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faa994a326a74892824eec2d26856312'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:47.363303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-b4d72e65-0313-498c-bf02-7633549fc79c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faa994a326a74892824eec2d26856312'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:47.373432 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-b4d72e65-0313-498c-bf02-7633549fc79c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faa994a326a74892824eec2d26856312'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:47.394643 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-b4d72e65-0313-498c-bf02-7633549fc79c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faa994a326a74892824eec2d26856312'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:47.396293 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-b4d72e65-0313-498c-bf02-7633549fc79c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faa994a326a74892824eec2d26856312'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:47.419894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 835/1659] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:05:47 2022] DELETE /identity/v3/users/faa994a326a74892824eec2d26856312 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:47.421954 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e814ba-d137-4195-88c0-1b9957624efd None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:47.422489 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e814ba-d137-4195-88c0-1b9957624efd None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:47.422574 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e814ba-d137-4195-88c0-1b9957624efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:47.493548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-88e814ba-d137-4195-88c0-1b9957624efd None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:47.495568 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-88e814ba-d137-4195-88c0-1b9957624efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:47.504218 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:47.692930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 825/1660] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:47.764774 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d72e65-0313-498c-bf02-7633549fc79c admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:47.766817 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b4d72e65-0313-498c-bf02-7633549fc79c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:47.961016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d72e65-0313-498c-bf02-7633549fc79c admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:47.967941 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3da1a-2a97-4bfc-be01-7ed3aa2701ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:47.969661 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3da1a-2a97-4bfc-be01-7ed3aa2701ed service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:47.970862 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3da1a-2a97-4bfc-be01-7ed3aa2701ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:47.972336 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3da1a-2a97-4bfc-be01-7ed3aa2701ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:47.997184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3da1a-2a97-4bfc-be01-7ed3aa2701ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:47.999062 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3da1a-2a97-4bfc-be01-7ed3aa2701ed service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:48.013127 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 836/1661] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:48.244675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e814ba-d137-4195-88c0-1b9957624efd None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:48.246644 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-88e814ba-d137-4195-88c0-1b9957624efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:48.270965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e814ba-d137-4195-88c0-1b9957624efd None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:48.278630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e6b22-9ab0-4f5c-a54a-b4206432f5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:48.279991 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e6b22-9ab0-4f5c-a54a-b4206432f5e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:48.280377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e6b22-9ab0-4f5c-a54a-b4206432f5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:48.281019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e6b22-9ab0-4f5c-a54a-b4206432f5e3 admin admin] PATH_INFO: `/v3/projects/c9496181d0314661a8645a5fc6f6f8a2` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:48.282656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e6b22-9ab0-4f5c-a54a-b4206432f5e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9496181d0314661a8645a5fc6f6f8a2)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:48.285134 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e6b22-9ab0-4f5c-a54a-b4206432f5e3 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:48.370215 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-440e6b22-9ab0-4f5c-a54a-b4206432f5e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9496181d0314661a8645a5fc6f6f8a2', 'request_id': 'req-440e6b22-9ab0-4f5c-a54a-b4206432f5e3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:48.387408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-440e6b22-9ab0-4f5c-a54a-b4206432f5e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9496181d0314661a8645a5fc6f6f8a2', 'request_id': 'req-440e6b22-9ab0-4f5c-a54a-b4206432f5e3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:48.411737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-440e6b22-9ab0-4f5c-a54a-b4206432f5e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9496181d0314661a8645a5fc6f6f8a2', 'request_id': 'req-440e6b22-9ab0-4f5c-a54a-b4206432f5e3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:48.455943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-440e6b22-9ab0-4f5c-a54a-b4206432f5e3 admin admin] The token cache is being invalidate because project c9496181d0314661a8645a5fc6f6f8a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:05:48.457064 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-440e6b22-9ab0-4f5c-a54a-b4206432f5e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:48.458121 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-440e6b22-9ab0-4f5c-a54a-b4206432f5e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:48.460877 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 826/1662] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:05:48 2022] DELETE /identity/v3/projects/c9496181d0314661a8645a5fc6f6f8a2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:48.466778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73e98d-33db-4bd0-a3f2-27b692e42e1c None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:48.467298 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73e98d-33db-4bd0-a3f2-27b692e42e1c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:48.467375 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73e98d-33db-4bd0-a3f2-27b692e42e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:48.510463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-7f73e98d-33db-4bd0-a3f2-27b692e42e1c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:48.511798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-7f73e98d-33db-4bd0-a3f2-27b692e42e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:48.640433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:48.704261 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 837/1663] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:48.706675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f0792-2467-40c4-85c8-af15f59112de None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:48.707044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f0792-2467-40c4-85c8-af15f59112de None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:48.707368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f0792-2467-40c4-85c8-af15f59112de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:48.762255 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-528f0792-2467-40c4-85c8-af15f59112de None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:48.763970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-528f0792-2467-40c4-85c8-af15f59112de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:48.783485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 827/1664] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:48.787528 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73e98d-33db-4bd0-a3f2-27b692e42e1c None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:48.806125 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73e98d-33db-4bd0-a3f2-27b692e42e1c None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:48.821082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93eb12-e4a3-4681-97d2-46b5c6ccc686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:48.824114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93eb12-e4a3-4681-97d2-46b5c6ccc686 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:48.824419 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93eb12-e4a3-4681-97d2-46b5c6ccc686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:48.828182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93eb12-e4a3-4681-97d2-46b5c6ccc686 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:48.829091 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93eb12-e4a3-4681-97d2-46b5c6ccc686 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:05:48.834604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93eb12-e4a3-4681-97d2-46b5c6ccc686 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:48.848896 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93eb12-e4a3-4681-97d2-46b5c6ccc686 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:48.879077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 838/1665] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:05:48.886695 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f0792-2467-40c4-85c8-af15f59112de None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:48.896548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93eb12-e4a3-4681-97d2-46b5c6ccc686 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:48.904818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-3e93eb12-e4a3-4681-97d2-46b5c6ccc686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:48.914453 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f0792-2467-40c4-85c8-af15f59112de None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:48.937105 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e340b-b5df-4277-be0d-5faad476f295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:48.938166 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e340b-b5df-4277-be0d-5faad476f295 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:48.940529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e340b-b5df-4277-be0d-5faad476f295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:48.940736 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93eb12-e4a3-4681-97d2-46b5c6ccc686 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:48.941018 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e340b-b5df-4277-be0d-5faad476f295 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:48.942680 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e340b-b5df-4277-be0d-5faad476f295 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:48.946025 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e340b-b5df-4277-be0d-5faad476f295 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:48.950634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75559e44-956d-4ec3-a0d4-eccfebbe1eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:48.951526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75559e44-956d-4ec3-a0d4-eccfebbe1eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:48.951811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75559e44-956d-4ec3-a0d4-eccfebbe1eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:48.952088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75559e44-956d-4ec3-a0d4-eccfebbe1eed admin admin] PATH_INFO: `/v3/projects/cde6ef220c0949119e7a9c559a6a93fb` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:48.953714 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75559e44-956d-4ec3-a0d4-eccfebbe1eed admin admin] RBAC: Authorizing `identity:delete_project(project_id=cde6ef220c0949119e7a9c559a6a93fb)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:48.954833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: WARNING py.warnings [None req-75559e44-956d-4ec3-a0d4-eccfebbe1eed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 22:05:48.954833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: warnings.warn(msg) Jul 12 22:05:48.954833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: Jul 12 22:05:48.961760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75559e44-956d-4ec3-a0d4-eccfebbe1eed admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:49.058554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 828/1666] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:48 2022] POST /identity/v3/projects => generated 379 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:49.059965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e340b-b5df-4277-be0d-5faad476f295 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:49.111673 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-75559e44-956d-4ec3-a0d4-eccfebbe1eed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cde6ef220c0949119e7a9c559a6a93fb', 'request_id': 'req-75559e44-956d-4ec3-a0d4-eccfebbe1eed'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:49.126373 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-75559e44-956d-4ec3-a0d4-eccfebbe1eed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cde6ef220c0949119e7a9c559a6a93fb', 'request_id': 'req-75559e44-956d-4ec3-a0d4-eccfebbe1eed'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:49.149249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-75559e44-956d-4ec3-a0d4-eccfebbe1eed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cde6ef220c0949119e7a9c559a6a93fb', 'request_id': 'req-75559e44-956d-4ec3-a0d4-eccfebbe1eed'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:49.191108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-75559e44-956d-4ec3-a0d4-eccfebbe1eed admin admin] The token cache is being invalidate because project cde6ef220c0949119e7a9c559a6a93fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:05:49.191868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-75559e44-956d-4ec3-a0d4-eccfebbe1eed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:49.201865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-75559e44-956d-4ec3-a0d4-eccfebbe1eed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:49.221889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 839/1667] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:05:48 2022] DELETE /identity/v3/projects/cde6ef220c0949119e7a9c559a6a93fb => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:49.221889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75559e44-956d-4ec3-a0d4-eccfebbe1eed admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:49.224578 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-75559e44-956d-4ec3-a0d4-eccfebbe1eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:49.262421 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e340b-b5df-4277-be0d-5faad476f295 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:49.277278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:49.410741 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8963b-47ca-464c-a13a-c00583360f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:49.411635 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8963b-47ca-464c-a13a-c00583360f95 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:49.411933 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8963b-47ca-464c-a13a-c00583360f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:49.412226 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8963b-47ca-464c-a13a-c00583360f95 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:49.412978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8963b-47ca-464c-a13a-c00583360f95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:49.427240 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8963b-47ca-464c-a13a-c00583360f95 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:49.439822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75559e44-956d-4ec3-a0d4-eccfebbe1eed admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:49.465472 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc362dfa-9970-47b7-afbe-9cbeb68614eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:49.466384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc362dfa-9970-47b7-afbe-9cbeb68614eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:49.466840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc362dfa-9970-47b7-afbe-9cbeb68614eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:49.467728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc362dfa-9970-47b7-afbe-9cbeb68614eb admin admin] PATH_INFO: `/v3/projects/f7d10482f6bb42ca9e7af85a7478e97c` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:49.470772 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc362dfa-9970-47b7-afbe-9cbeb68614eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7d10482f6bb42ca9e7af85a7478e97c)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:49.476069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc362dfa-9970-47b7-afbe-9cbeb68614eb admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:49.497868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 829/1668] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:49 2022] POST /identity/v3/users => generated 290 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:49.498886 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8963b-47ca-464c-a13a-c00583360f95 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:49.506391 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-76b8963b-47ca-464c-a13a-c00583360f95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:49.545905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8963b-47ca-464c-a13a-c00583360f95 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:49.570301 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139142d-b3a7-4a6d-8f99-3e473cb8ec3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:49.571238 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139142d-b3a7-4a6d-8f99-3e473cb8ec3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:49.572339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139142d-b3a7-4a6d-8f99-3e473cb8ec3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:49.572645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139142d-b3a7-4a6d-8f99-3e473cb8ec3c admin admin] PATH_INFO: `/v3/users/41b6585328bc4134a67820b5114c19c3` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:49.575329 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139142d-b3a7-4a6d-8f99-3e473cb8ec3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=41b6585328bc4134a67820b5114c19c3)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:49.599449 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139142d-b3a7-4a6d-8f99-3e473cb8ec3c admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:49.659401 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dc362dfa-9970-47b7-afbe-9cbeb68614eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7d10482f6bb42ca9e7af85a7478e97c', 'request_id': 'req-dc362dfa-9970-47b7-afbe-9cbeb68614eb'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:49.693438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dc362dfa-9970-47b7-afbe-9cbeb68614eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7d10482f6bb42ca9e7af85a7478e97c', 'request_id': 'req-dc362dfa-9970-47b7-afbe-9cbeb68614eb'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:49.704478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:49.718247 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dc362dfa-9970-47b7-afbe-9cbeb68614eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7d10482f6bb42ca9e7af85a7478e97c', 'request_id': 'req-dc362dfa-9970-47b7-afbe-9cbeb68614eb'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:49.752780 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dc362dfa-9970-47b7-afbe-9cbeb68614eb admin admin] The token cache is being invalidate because project f7d10482f6bb42ca9e7af85a7478e97c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:05:49.753085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dc362dfa-9970-47b7-afbe-9cbeb68614eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:49.761146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dc362dfa-9970-47b7-afbe-9cbeb68614eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:49.772573 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 840/1669] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:05:49 2022] DELETE /identity/v3/projects/f7d10482f6bb42ca9e7af85a7478e97c => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:49.777063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1c0b3-65ad-4f69-9812-c04aa20ef1dd None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:49.777602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1c0b3-65ad-4f69-9812-c04aa20ef1dd None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:49.780318 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1c0b3-65ad-4f69-9812-c04aa20ef1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:49.785668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f139142d-b3a7-4a6d-8f99-3e473cb8ec3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41b6585328bc4134a67820b5114c19c3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:49.823654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f139142d-b3a7-4a6d-8f99-3e473cb8ec3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41b6585328bc4134a67820b5114c19c3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:49.844462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f139142d-b3a7-4a6d-8f99-3e473cb8ec3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41b6585328bc4134a67820b5114c19c3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:49.845822 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f139142d-b3a7-4a6d-8f99-3e473cb8ec3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41b6585328bc4134a67820b5114c19c3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:49.847548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f139142d-b3a7-4a6d-8f99-3e473cb8ec3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41b6585328bc4134a67820b5114c19c3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:49.879680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-ebe1c0b3-65ad-4f69-9812-c04aa20ef1dd None None] MFA Rules not processed for user `9cdeaf848c1f4fd8b110f2ceeadd5b62`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:49.881074 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ebe1c0b3-65ad-4f69-9812-c04aa20ef1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:49.916955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 830/1670] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:05:49 2022] DELETE /identity/v3/users/41b6585328bc4134a67820b5114c19c3 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:49.918119 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139142d-b3a7-4a6d-8f99-3e473cb8ec3c admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:49.922551 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-f139142d-b3a7-4a6d-8f99-3e473cb8ec3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:50.062887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 841/1671] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:49 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:50.064145 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1c0b3-65ad-4f69-9812-c04aa20ef1dd None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:50.069696 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ebe1c0b3-65ad-4f69-9812-c04aa20ef1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:50.082728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139142d-b3a7-4a6d-8f99-3e473cb8ec3c admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:50.092920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef523ea-c241-4128-b25a-849a6b513dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:50.093906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef523ea-c241-4128-b25a-849a6b513dac admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:50.094383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef523ea-c241-4128-b25a-849a6b513dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:50.094804 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef523ea-c241-4128-b25a-849a6b513dac admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:50.095700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef523ea-c241-4128-b25a-849a6b513dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:50.110841 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef523ea-c241-4128-b25a-849a6b513dac admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:50.137390 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 831/1672] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:50.143520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef523ea-c241-4128-b25a-849a6b513dac admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:50.175102 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef523ea-c241-4128-b25a-849a6b513dac admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:50.180793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0074aab-5fcb-4133-8f37-6d4c959cb233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:50.181498 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0074aab-5fcb-4133-8f37-6d4c959cb233 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:50.181794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0074aab-5fcb-4133-8f37-6d4c959cb233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:50.182080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0074aab-5fcb-4133-8f37-6d4c959cb233 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:50.182829 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0074aab-5fcb-4133-8f37-6d4c959cb233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:50.184493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0074aab-5fcb-4133-8f37-6d4c959cb233 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:50.198432 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:50.215269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 832/1673] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:50.216242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0074aab-5fcb-4133-8f37-6d4c959cb233 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:50.237066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0074aab-5fcb-4133-8f37-6d4c959cb233 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:50.247347 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6678ce-fd33-4827-b45f-85c92e15f186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:50.249033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6678ce-fd33-4827-b45f-85c92e15f186 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:50.249487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6678ce-fd33-4827-b45f-85c92e15f186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:50.249725 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6678ce-fd33-4827-b45f-85c92e15f186 admin admin] PATH_INFO: `/v3/projects/e2bd3999baa445ddbe68ccd26787bba5/users/02158c23639543618aa319de5be4aadc/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:50.269076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1c0b3-65ad-4f69-9812-c04aa20ef1dd None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:50.277134 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6678ce-fd33-4827-b45f-85c92e15f186 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2bd3999baa445ddbe68ccd26787bba5, user_id=02158c23639543618aa319de5be4aadc, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:50.281457 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6678ce-fd33-4827-b45f-85c92e15f186 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:50.288771 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd4908-9832-4979-b043-cec3eef9f28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:50.291987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd4908-9832-4979-b043-cec3eef9f28a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:50.292751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd4908-9832-4979-b043-cec3eef9f28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:50.305348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd4908-9832-4979-b043-cec3eef9f28a admin admin] PATH_INFO: `/v3/users/8dfeeaffaf884f95bb0f8f30583a8fd3` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:50.311888 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd4908-9832-4979-b043-cec3eef9f28a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dfeeaffaf884f95bb0f8f30583a8fd3)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:50.320364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd4908-9832-4979-b043-cec3eef9f28a admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:50.324547 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 833/1674] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:50 2022] PUT /identity/v3/projects/e2bd3999baa445ddbe68ccd26787bba5/users/02158c23639543618aa319de5be4aadc/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:50.326334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac29ec-2bd8-4587-9a3e-dba7c9f47a5b None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:50.326694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac29ec-2bd8-4587-9a3e-dba7c9f47a5b None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:50.327044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac29ec-2bd8-4587-9a3e-dba7c9f47a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:50.366531 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-95ac29ec-2bd8-4587-9a3e-dba7c9f47a5b None None] MFA Rules not processed for user `02158c23639543618aa319de5be4aadc`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:50.367930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-95ac29ec-2bd8-4587-9a3e-dba7c9f47a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:50.487392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-95dd4908-9832-4979-b043-cec3eef9f28a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dfeeaffaf884f95bb0f8f30583a8fd3'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:50.525771 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-95dd4908-9832-4979-b043-cec3eef9f28a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dfeeaffaf884f95bb0f8f30583a8fd3'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:50.550289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-95dd4908-9832-4979-b043-cec3eef9f28a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dfeeaffaf884f95bb0f8f30583a8fd3'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:50.557369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 834/1675] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:50.558344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:50.558478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac29ec-2bd8-4587-9a3e-dba7c9f47a5b None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:50.577806 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-95dd4908-9832-4979-b043-cec3eef9f28a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dfeeaffaf884f95bb0f8f30583a8fd3'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:50.580503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-95dd4908-9832-4979-b043-cec3eef9f28a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dfeeaffaf884f95bb0f8f30583a8fd3'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:50.604976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 842/1676] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:05:50 2022] DELETE /identity/v3/users/8dfeeaffaf884f95bb0f8f30583a8fd3 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:50.702213 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac29ec-2bd8-4587-9a3e-dba7c9f47a5b None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:50.785770 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b81fb7-ca26-4db0-ad0a-c5c892c7576a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:50.786755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b81fb7-ca26-4db0-ad0a-c5c892c7576a admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:50.787049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b81fb7-ca26-4db0-ad0a-c5c892c7576a admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:50.787422 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b81fb7-ca26-4db0-ad0a-c5c892c7576a admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:50.788157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b81fb7-ca26-4db0-ad0a-c5c892c7576a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:50.791017 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b81fb7-ca26-4db0-ad0a-c5c892c7576a admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:50.817638 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:50.837668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 835/1677] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:50 2022] POST /identity/v3/projects => generated 379 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:50.841271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd4908-9832-4979-b043-cec3eef9f28a admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:50.844457 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-95dd4908-9832-4979-b043-cec3eef9f28a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:50.876060 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd4908-9832-4979-b043-cec3eef9f28a admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:50.989999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ae0d5-49ed-4957-bf94-12006347454f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:50.992451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ae0d5-49ed-4957-bf94-12006347454f admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:50.992993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ae0d5-49ed-4957-bf94-12006347454f admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:50.993989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ae0d5-49ed-4957-bf94-12006347454f admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:50.996015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ae0d5-49ed-4957-bf94-12006347454f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:51.004267 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ae0d5-49ed-4957-bf94-12006347454f admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:51.060591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 843/1678] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:50 2022] POST /identity/v3/users => generated 296 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:51.068075 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b81fb7-ca26-4db0-ad0a-c5c892c7576a admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:51.089358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b81fb7-ca26-4db0-ad0a-c5c892c7576a admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:51.105743 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd691c28-aaa9-4fdd-a501-40d7ebeda7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:51.108099 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd691c28-aaa9-4fdd-a501-40d7ebeda7ff admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:51.109051 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd691c28-aaa9-4fdd-a501-40d7ebeda7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:51.109413 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd691c28-aaa9-4fdd-a501-40d7ebeda7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:51.110249 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd691c28-aaa9-4fdd-a501-40d7ebeda7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:51.119666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd691c28-aaa9-4fdd-a501-40d7ebeda7ff admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:51.139955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 836/1679] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:51.146290 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ae0d5-49ed-4957-bf94-12006347454f admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:51.183043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ae0d5-49ed-4957-bf94-12006347454f admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:51.199845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c353cd-92da-4871-9668-818758db9521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:51.200937 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c353cd-92da-4871-9668-818758db9521 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:51.203142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c353cd-92da-4871-9668-818758db9521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:51.203590 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c353cd-92da-4871-9668-818758db9521 admin admin] PATH_INFO: `/v3/projects/de74f612e99b4b3e8bc5d106db6a03c2/users/f2de12ca8a2c470095facf741e0acabc/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:51.262275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c353cd-92da-4871-9668-818758db9521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de74f612e99b4b3e8bc5d106db6a03c2, user_id=f2de12ca8a2c470095facf741e0acabc, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:51.271879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c353cd-92da-4871-9668-818758db9521 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:51.316743 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 844/1680] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:51 2022] PUT /identity/v3/projects/de74f612e99b4b3e8bc5d106db6a03c2/users/f2de12ca8a2c470095facf741e0acabc/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:51.323257 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd691c28-aaa9-4fdd-a501-40d7ebeda7ff admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:51.340376 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd691c28-aaa9-4fdd-a501-40d7ebeda7ff admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:51.401872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c353cd-92da-4871-9668-818758db9521 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:51.405193 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-22c353cd-92da-4871-9668-818758db9521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:51.521912 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6fce1-f922-48dd-9174-8cb186374772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:51.522628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6fce1-f922-48dd-9174-8cb186374772 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:51.522917 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6fce1-f922-48dd-9174-8cb186374772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:51.523204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6fce1-f922-48dd-9174-8cb186374772 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:51.523936 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6fce1-f922-48dd-9174-8cb186374772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:51.532987 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6fce1-f922-48dd-9174-8cb186374772 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:51.573259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 837/1681] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:51.574909 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6fce1-f922-48dd-9174-8cb186374772 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:51.601869 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6fce1-f922-48dd-9174-8cb186374772 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:51.619786 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb921c6d-5059-4d94-9029-cf99c7084b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:51.620721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb921c6d-5059-4d94-9029-cf99c7084b01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:51.624603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb921c6d-5059-4d94-9029-cf99c7084b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:51.629044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb921c6d-5059-4d94-9029-cf99c7084b01 admin admin] PATH_INFO: `/v3/projects/de74f612e99b4b3e8bc5d106db6a03c2/users/f2de12ca8a2c470095facf741e0acabc/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:51.636274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb921c6d-5059-4d94-9029-cf99c7084b01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de74f612e99b4b3e8bc5d106db6a03c2, user_id=f2de12ca8a2c470095facf741e0acabc, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:51.639339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb921c6d-5059-4d94-9029-cf99c7084b01 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:51.651823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c353cd-92da-4871-9668-818758db9521 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:51.667257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127b437-dd84-4a31-bc61-71ce48552f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:51.667257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127b437-dd84-4a31-bc61-71ce48552f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:51.667581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127b437-dd84-4a31-bc61-71ce48552f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:51.667939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127b437-dd84-4a31-bc61-71ce48552f7c admin admin] PATH_INFO: `/v3/projects/f98dc7df181e45d98c1b23e061d556a3` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:51.673940 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127b437-dd84-4a31-bc61-71ce48552f7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f98dc7df181e45d98c1b23e061d556a3)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:51.674294 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 838/1682] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:51 2022] PUT /identity/v3/projects/de74f612e99b4b3e8bc5d106db6a03c2/users/f2de12ca8a2c470095facf741e0acabc/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:51.685587 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127b437-dd84-4a31-bc61-71ce48552f7c admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:51.685794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb921c6d-5059-4d94-9029-cf99c7084b01 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:51.685794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-cb921c6d-5059-4d94-9029-cf99c7084b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:51.812741 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-a127b437-dd84-4a31-bc61-71ce48552f7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f98dc7df181e45d98c1b23e061d556a3', 'request_id': 'req-a127b437-dd84-4a31-bc61-71ce48552f7c'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:51.825750 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-a127b437-dd84-4a31-bc61-71ce48552f7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f98dc7df181e45d98c1b23e061d556a3', 'request_id': 'req-a127b437-dd84-4a31-bc61-71ce48552f7c'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:51.831495 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:51.842398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-a127b437-dd84-4a31-bc61-71ce48552f7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f98dc7df181e45d98c1b23e061d556a3', 'request_id': 'req-a127b437-dd84-4a31-bc61-71ce48552f7c'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:51.867090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-a127b437-dd84-4a31-bc61-71ce48552f7c admin admin] The token cache is being invalidate because project f98dc7df181e45d98c1b23e061d556a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:05:51.868603 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-a127b437-dd84-4a31-bc61-71ce48552f7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:51.868854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb921c6d-5059-4d94-9029-cf99c7084b01 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:51.870175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-a127b437-dd84-4a31-bc61-71ce48552f7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:51.872187 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 845/1683] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:05:51 2022] DELETE /identity/v3/projects/f98dc7df181e45d98c1b23e061d556a3 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:51.874046 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2204ec9-e86a-4b4a-9ade-e8b4dc78a22a None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:51.874645 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2204ec9-e86a-4b4a-9ade-e8b4dc78a22a None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:51.875018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2204ec9-e86a-4b4a-9ade-e8b4dc78a22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:51.970092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-b2204ec9-e86a-4b4a-9ade-e8b4dc78a22a None None] MFA Rules not processed for user `f2de12ca8a2c470095facf741e0acabc`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:51.971411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b2204ec9-e86a-4b4a-9ade-e8b4dc78a22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:51.981137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69c4ba-5e0f-489d-8ad1-b116d4d6b2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:51.985437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69c4ba-5e0f-489d-8ad1-b116d4d6b2ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:51.986430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69c4ba-5e0f-489d-8ad1-b116d4d6b2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:51.987204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69c4ba-5e0f-489d-8ad1-b116d4d6b2ed admin admin] PATH_INFO: `/v3/users/e7d835605f074ba885baffa7d023f943` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:51.989109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69c4ba-5e0f-489d-8ad1-b116d4d6b2ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7d835605f074ba885baffa7d023f943)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:51.991710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69c4ba-5e0f-489d-8ad1-b116d4d6b2ed admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:52.027124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:52.076992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0c69c4ba-5e0f-489d-8ad1-b116d4d6b2ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7d835605f074ba885baffa7d023f943'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:52.090549 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0c69c4ba-5e0f-489d-8ad1-b116d4d6b2ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7d835605f074ba885baffa7d023f943'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:52.106567 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0c69c4ba-5e0f-489d-8ad1-b116d4d6b2ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7d835605f074ba885baffa7d023f943'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:52.107523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0c69c4ba-5e0f-489d-8ad1-b116d4d6b2ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7d835605f074ba885baffa7d023f943'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:52.111558 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0c69c4ba-5e0f-489d-8ad1-b116d4d6b2ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7d835605f074ba885baffa7d023f943'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:52.146489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 846/1684] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:52.151823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6753003-40a4-4a97-88c8-6dd6f12e2bca None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:52.153741 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6753003-40a4-4a97-88c8-6dd6f12e2bca None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:52.154437 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6753003-40a4-4a97-88c8-6dd6f12e2bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:52.172855 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 839/1685] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:05:51 2022] DELETE /identity/v3/users/e7d835605f074ba885baffa7d023f943 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:52.173991 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69c4ba-5e0f-489d-8ad1-b116d4d6b2ed admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:52.178098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0c69c4ba-5e0f-489d-8ad1-b116d4d6b2ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:52.234576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-e6753003-40a4-4a97-88c8-6dd6f12e2bca None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:52.240366 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e6753003-40a4-4a97-88c8-6dd6f12e2bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:52.279621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69c4ba-5e0f-489d-8ad1-b116d4d6b2ed admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:52.285560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa4932-c90d-4f52-aec0-c20cd695597e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:52.286379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa4932-c90d-4f52-aec0-c20cd695597e service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:52.286782 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa4932-c90d-4f52-aec0-c20cd695597e service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:52.287217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa4932-c90d-4f52-aec0-c20cd695597e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:52.431864 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 847/1686] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:52.434931 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9df337-01e4-48d2-8bf7-5826370bfdbd None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:52.435227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9df337-01e4-48d2-8bf7-5826370bfdbd None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:52.435556 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9df337-01e4-48d2-8bf7-5826370bfdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:52.486250 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-0b9df337-01e4-48d2-8bf7-5826370bfdbd None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:52.487411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0b9df337-01e4-48d2-8bf7-5826370bfdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:52.490809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa4932-c90d-4f52-aec0-c20cd695597e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:52.492366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa4932-c90d-4f52-aec0-c20cd695597e service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:52.509310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 848/1687] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:52.510796 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda63e6b-37db-4181-a2a1-d5916bf927f3 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:52.511082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda63e6b-37db-4181-a2a1-d5916bf927f3 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:52.511359 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda63e6b-37db-4181-a2a1-d5916bf927f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:52.517302 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 840/1688] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:52.535902 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-cda63e6b-37db-4181-a2a1-d5916bf927f3 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:52.537088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-cda63e6b-37db-4181-a2a1-d5916bf927f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:52.550345 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 849/1689] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:52.555166 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa4932-c90d-4f52-aec0-c20cd695597e service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:52.569129 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda63e6b-37db-4181-a2a1-d5916bf927f3 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:52.573440 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa4932-c90d-4f52-aec0-c20cd695597e service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:52.574973 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-cda63e6b-37db-4181-a2a1-d5916bf927f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:52.599752 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784529ef-ebf4-48db-af4a-4a9b3e2ce3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:52.600442 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784529ef-ebf4-48db-af4a-4a9b3e2ce3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:52.600740 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784529ef-ebf4-48db-af4a-4a9b3e2ce3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:52.601702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784529ef-ebf4-48db-af4a-4a9b3e2ce3f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:52.602372 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784529ef-ebf4-48db-af4a-4a9b3e2ce3f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:05:52.603463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784529ef-ebf4-48db-af4a-4a9b3e2ce3f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:52.606012 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784529ef-ebf4-48db-af4a-4a9b3e2ce3f2 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:52.636835 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 841/1690] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:05:52.636835 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784529ef-ebf4-48db-af4a-4a9b3e2ce3f2 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:52.639085 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-784529ef-ebf4-48db-af4a-4a9b3e2ce3f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:52.671199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784529ef-ebf4-48db-af4a-4a9b3e2ce3f2 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:52.689530 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:52.690537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:52.691511 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:52.691859 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2 admin admin] PATH_INFO: `/v3/projects/453ee4bddca64aa5b4fd56447be26046` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:52.693556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=453ee4bddca64aa5b4fd56447be26046)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:52.696957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:52.719147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda63e6b-37db-4181-a2a1-d5916bf927f3 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:52.723834 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825880d-7a10-462f-b41e-8a61657c4003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:52.724848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825880d-7a10-462f-b41e-8a61657c4003 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:52.725973 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825880d-7a10-462f-b41e-8a61657c4003 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:52.728311 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825880d-7a10-462f-b41e-8a61657c4003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:52.768249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825880d-7a10-462f-b41e-8a61657c4003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:52.794298 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825880d-7a10-462f-b41e-8a61657c4003 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:52.803136 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:52.842385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '453ee4bddca64aa5b4fd56447be26046', 'request_id': 'req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:52.849288 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 850/1691] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:05:52.850286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825880d-7a10-462f-b41e-8a61657c4003 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:52.858126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:52.860413 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '453ee4bddca64aa5b4fd56447be26046', 'request_id': 'req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:52.890200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '453ee4bddca64aa5b4fd56447be26046', 'request_id': 'req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:52.890863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825880d-7a10-462f-b41e-8a61657c4003 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:52.913375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2 admin admin] The token cache is being invalidate because project 453ee4bddca64aa5b4fd56447be26046 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:05:52.913764 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:52.916231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:52.918281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 842/1692] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:05:52 2022] DELETE /identity/v3/projects/453ee4bddca64aa5b4fd56447be26046 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:52.919069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:52.925724 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:53.032424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368ec8c-18cc-46d4-b107-997e8029edae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:53.033551 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368ec8c-18cc-46d4-b107-997e8029edae admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:53.033900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368ec8c-18cc-46d4-b107-997e8029edae admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:53.034194 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368ec8c-18cc-46d4-b107-997e8029edae admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:53.035079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368ec8c-18cc-46d4-b107-997e8029edae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:53.043176 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368ec8c-18cc-46d4-b107-997e8029edae admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:53.066607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce7624-e2d8-47b2-ae89-87b0a669bdd2 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:53.080438 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e868a7-f68e-4700-a68b-d6a28a9affa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:53.081518 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e868a7-f68e-4700-a68b-d6a28a9affa0 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:53.082439 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e868a7-f68e-4700-a68b-d6a28a9affa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:53.082816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e868a7-f68e-4700-a68b-d6a28a9affa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:53.085048 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a4e868a7-f68e-4700-a68b-d6a28a9affa0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:53.101543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 851/1693] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:52 2022] POST /identity/v3/projects => generated 385 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:53.103733 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71814d25-da90-4afe-9109-30d0fb05ba9c None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:53.104034 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71814d25-da90-4afe-9109-30d0fb05ba9c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:53.104316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71814d25-da90-4afe-9109-30d0fb05ba9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:53.133595 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-71814d25-da90-4afe-9109-30d0fb05ba9c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:53.135040 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-71814d25-da90-4afe-9109-30d0fb05ba9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:53.227319 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e868a7-f68e-4700-a68b-d6a28a9affa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:53.231858 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e868a7-f68e-4700-a68b-d6a28a9affa0 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:53.236988 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 852/1694] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:53.237977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71814d25-da90-4afe-9109-30d0fb05ba9c None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:53.239422 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-71814d25-da90-4afe-9109-30d0fb05ba9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:53.249189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 843/1695] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:53.251536 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0d8b5-725e-4c88-9c30-dbee52f8e0de None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:53.252156 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0d8b5-725e-4c88-9c30-dbee52f8e0de None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:53.252535 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0d8b5-725e-4c88-9c30-dbee52f8e0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:53.254459 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71814d25-da90-4afe-9109-30d0fb05ba9c None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:53.268142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd1676-3cee-415c-b214-67618975d67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:53.268846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd1676-3cee-415c-b214-67618975d67b admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:53.269821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd1676-3cee-415c-b214-67618975d67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:53.270174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd1676-3cee-415c-b214-67618975d67b admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:53.270971 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd1676-3cee-415c-b214-67618975d67b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:53.273504 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd1676-3cee-415c-b214-67618975d67b admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:53.278093 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-cdc0d8b5-725e-4c88-9c30-dbee52f8e0de None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:53.279477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-cdc0d8b5-725e-4c88-9c30-dbee52f8e0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:53.290807 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 844/1696] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:53.295177 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0d8b5-725e-4c88-9c30-dbee52f8e0de None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:53.308686 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 853/1697] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:53 2022] POST /identity/v3/users => generated 293 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:53.313865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd1676-3cee-415c-b214-67618975d67b admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:53.317631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0d8b5-725e-4c88-9c30-dbee52f8e0de None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:53.330212 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd1e03-bd0e-4bb5-afcb-a2290677f3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:53.332113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd1e03-bd0e-4bb5-afcb-a2290677f3dc admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:53.332899 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd1e03-bd0e-4bb5-afcb-a2290677f3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:53.333668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd1e03-bd0e-4bb5-afcb-a2290677f3dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:53.335654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd1e03-bd0e-4bb5-afcb-a2290677f3dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:05:53.337363 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd1e03-bd0e-4bb5-afcb-a2290677f3dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:53.339080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd1676-3cee-415c-b214-67618975d67b admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:53.341417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd1e03-bd0e-4bb5-afcb-a2290677f3dc admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:53.354029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89341b8d-ab7a-447a-9ff8-c5a13b2ec35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:53.354822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89341b8d-ab7a-447a-9ff8-c5a13b2ec35e admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:53.355333 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89341b8d-ab7a-447a-9ff8-c5a13b2ec35e admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:53.355719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89341b8d-ab7a-447a-9ff8-c5a13b2ec35e admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:53.356814 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89341b8d-ab7a-447a-9ff8-c5a13b2ec35e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:53.361726 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89341b8d-ab7a-447a-9ff8-c5a13b2ec35e admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:53.375367 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 845/1698] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 22:05:53.379066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd1e03-bd0e-4bb5-afcb-a2290677f3dc admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:53.388982 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 854/1699] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:05:53.392743 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89341b8d-ab7a-447a-9ff8-c5a13b2ec35e admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:53.406128 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd1e03-bd0e-4bb5-afcb-a2290677f3dc admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:53.411786 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89341b8d-ab7a-447a-9ff8-c5a13b2ec35e admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:53.420799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc0ba4-64e5-419a-ae2e-f0abfe995427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:53.421483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc0ba4-64e5-419a-ae2e-f0abfe995427 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:53.421771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc0ba4-64e5-419a-ae2e-f0abfe995427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:53.422055 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc0ba4-64e5-419a-ae2e-f0abfe995427 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:53.423127 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc0ba4-64e5-419a-ae2e-f0abfe995427 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:53.423387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293edd8-b950-4048-a840-cdb6cdc2c6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:53.424145 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293edd8-b950-4048-a840-cdb6cdc2c6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:53.424496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293edd8-b950-4048-a840-cdb6cdc2c6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:53.425468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293edd8-b950-4048-a840-cdb6cdc2c6b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:53.427217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293edd8-b950-4048-a840-cdb6cdc2c6b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:53.428996 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293edd8-b950-4048-a840-cdb6cdc2c6b0 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:53.430381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc0ba4-64e5-419a-ae2e-f0abfe995427 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:53.468331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 855/1700] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:53.477371 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293edd8-b950-4048-a840-cdb6cdc2c6b0 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:53.498955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 846/1701] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:53 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:53.500006 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc0ba4-64e5-419a-ae2e-f0abfe995427 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:53.509899 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293edd8-b950-4048-a840-cdb6cdc2c6b0 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:53.568702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:53.625146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc0ba4-64e5-419a-ae2e-f0abfe995427 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:53.631398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:53.632733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac246d6-059b-4929-a12f-55af0477a956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:53.633557 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac246d6-059b-4929-a12f-55af0477a956 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:53.634865 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac246d6-059b-4929-a12f-55af0477a956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:53.635431 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac246d6-059b-4929-a12f-55af0477a956 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:53.637633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac246d6-059b-4929-a12f-55af0477a956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:53.641557 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac246d6-059b-4929-a12f-55af0477a956 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:53.653232 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef69f8e-7f78-47b4-98c1-4e212f277f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:53.655832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef69f8e-7f78-47b4-98c1-4e212f277f29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:53.657503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef69f8e-7f78-47b4-98c1-4e212f277f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:53.658174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef69f8e-7f78-47b4-98c1-4e212f277f29 admin admin] PATH_INFO: `/v3/projects/f46254a3fea94bf19cacba55988e4c49/users/73ae5f51260447c2b2799d6ff88841fc/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:53.685626 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 847/1702] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:53 2022] POST /identity/v3/users => generated 291 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:53.686799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac246d6-059b-4929-a12f-55af0477a956 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:53.695415 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef69f8e-7f78-47b4-98c1-4e212f277f29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f46254a3fea94bf19cacba55988e4c49, user_id=73ae5f51260447c2b2799d6ff88841fc, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:53.702606 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac246d6-059b-4929-a12f-55af0477a956 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:53.705180 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef69f8e-7f78-47b4-98c1-4e212f277f29 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:53.722257 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b580cb2-9710-45ac-bf2e-cc798bc732fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:53.723101 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b580cb2-9710-45ac-bf2e-cc798bc732fc admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:53.723619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b580cb2-9710-45ac-bf2e-cc798bc732fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:53.724513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b580cb2-9710-45ac-bf2e-cc798bc732fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:53.727711 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b580cb2-9710-45ac-bf2e-cc798bc732fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:53.733105 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b580cb2-9710-45ac-bf2e-cc798bc732fc admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:53.733670 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 856/1703] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:53 2022] PUT /identity/v3/projects/f46254a3fea94bf19cacba55988e4c49/users/73ae5f51260447c2b2799d6ff88841fc/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:53.752004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad185b-2a0a-4b6d-82d8-aea0ea423757 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:53.754812 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad185b-2a0a-4b6d-82d8-aea0ea423757 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:53.755449 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad185b-2a0a-4b6d-82d8-aea0ea423757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:53.760375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 848/1704] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:53.764657 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b580cb2-9710-45ac-bf2e-cc798bc732fc admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:53.776033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b580cb2-9710-45ac-bf2e-cc798bc732fc admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:53.821852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-38ad185b-2a0a-4b6d-82d8-aea0ea423757 None None] MFA Rules not processed for user `73ae5f51260447c2b2799d6ff88841fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:53.828725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-38ad185b-2a0a-4b6d-82d8-aea0ea423757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:53.860486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e0648-e7ac-400f-a13f-8061e65d76d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:53.861439 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e0648-e7ac-400f-a13f-8061e65d76d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:53.861901 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e0648-e7ac-400f-a13f-8061e65d76d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:53.862554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e0648-e7ac-400f-a13f-8061e65d76d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:53.863514 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e0648-e7ac-400f-a13f-8061e65d76d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:53.865401 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e0648-e7ac-400f-a13f-8061e65d76d5 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:53.877695 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 849/1705] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:05:53.884456 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e0648-e7ac-400f-a13f-8061e65d76d5 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:53.914120 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e0648-e7ac-400f-a13f-8061e65d76d5 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:53.931146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b385018-5e94-474f-918d-dd24e42d4dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:53.932742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b385018-5e94-474f-918d-dd24e42d4dd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:53.933169 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b385018-5e94-474f-918d-dd24e42d4dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:53.933547 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b385018-5e94-474f-918d-dd24e42d4dd8 admin admin] PATH_INFO: `/v3/projects/4d6bcba1180b438ab9c25e01b3967728/users/23873b8219414fdfa5ce15f7e7aab99b/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:53.974577 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b385018-5e94-474f-918d-dd24e42d4dd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d6bcba1180b438ab9c25e01b3967728, user_id=23873b8219414fdfa5ce15f7e7aab99b, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:53.982318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b385018-5e94-474f-918d-dd24e42d4dd8 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:54.011954 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 850/1706] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:53 2022] PUT /identity/v3/projects/4d6bcba1180b438ab9c25e01b3967728/users/23873b8219414fdfa5ce15f7e7aab99b/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:05:54.013732 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922ffb2-50a0-4af3-bb96-031ab2d7aa9a None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:54.014130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922ffb2-50a0-4af3-bb96-031ab2d7aa9a None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:54.014496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922ffb2-50a0-4af3-bb96-031ab2d7aa9a None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:54.021114 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 851/1707] 10.222.0.44 () {54 vars in 921 bytes} [Tue Jul 12 22:05:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:05:54.023481 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b99f38-4202-4f6f-b6b0-55955626ef7d None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:54.023902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b99f38-4202-4f6f-b6b0-55955626ef7d None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:54.024310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b99f38-4202-4f6f-b6b0-55955626ef7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:54.083625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-e8b99f38-4202-4f6f-b6b0-55955626ef7d None None] MFA Rules not processed for user `23873b8219414fdfa5ce15f7e7aab99b`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:54.085220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e8b99f38-4202-4f6f-b6b0-55955626ef7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:54.088677 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 857/1708] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:05:54.093703 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad185b-2a0a-4b6d-82d8-aea0ea423757 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:54.112084 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad185b-2a0a-4b6d-82d8-aea0ea423757 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:54.257765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de5902-971a-4e00-a575-2cf1ef0665da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:54.258840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de5902-971a-4e00-a575-2cf1ef0665da admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:54.259191 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de5902-971a-4e00-a575-2cf1ef0665da admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:54.259602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de5902-971a-4e00-a575-2cf1ef0665da admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:54.260326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de5902-971a-4e00-a575-2cf1ef0665da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:54.262315 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de5902-971a-4e00-a575-2cf1ef0665da admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:54.272927 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 852/1709] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:54.275965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06c96d-6efc-4b6b-b79e-c4a5c5874fee None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:54.276272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06c96d-6efc-4b6b-b79e-c4a5c5874fee None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:54.276738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06c96d-6efc-4b6b-b79e-c4a5c5874fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:54.324154 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-2f06c96d-6efc-4b6b-b79e-c4a5c5874fee None None] MFA Rules not processed for user `a1f1ef52f8eb42a59a50a69873b9ffaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:54.325447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-2f06c96d-6efc-4b6b-b79e-c4a5c5874fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:54.330750 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 858/1710] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:54 2022] POST /identity/v3/projects => generated 387 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:54.332652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de5902-971a-4e00-a575-2cf1ef0665da admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:54.351066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:54.444891 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:54.485530 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de5902-971a-4e00-a575-2cf1ef0665da admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:54.500996 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53cdae-c8ea-4078-94dc-6ecc3e7df218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:54.501677 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53cdae-c8ea-4078-94dc-6ecc3e7df218 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:54.502050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53cdae-c8ea-4078-94dc-6ecc3e7df218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:54.502497 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53cdae-c8ea-4078-94dc-6ecc3e7df218 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:54.503271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53cdae-c8ea-4078-94dc-6ecc3e7df218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:54.506063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53cdae-c8ea-4078-94dc-6ecc3e7df218 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:54.525638 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 853/1711] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Jul 12 22:05:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:54.526251 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06c96d-6efc-4b6b-b79e-c4a5c5874fee None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:54.553044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06c96d-6efc-4b6b-b79e-c4a5c5874fee None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:54.563987 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c42e16-86d7-491a-9381-07433058efee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:54.564459 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 859/1712] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:54 2022] POST /identity/v3/projects => generated 381 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:54.564846 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c42e16-86d7-491a-9381-07433058efee admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:54.565387 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c42e16-86d7-491a-9381-07433058efee admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:54.565736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c42e16-86d7-491a-9381-07433058efee admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:54.566573 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c42e16-86d7-491a-9381-07433058efee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:54.568592 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c42e16-86d7-491a-9381-07433058efee admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:54.570949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9c833-2914-419e-aa66-9c1685a64186 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:54.572507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9c833-2914-419e-aa66-9c1685a64186 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:54.574518 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9c833-2914-419e-aa66-9c1685a64186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:54.626823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 854/1713] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:54 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:54.630431 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c42e16-86d7-491a-9381-07433058efee admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:54.655546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-caa9c833-2914-419e-aa66-9c1685a64186 None None] MFA Rules not processed for user `85dea53a4a034f5981b638bf92f2cf3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:54.656845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-caa9c833-2914-419e-aa66-9c1685a64186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:54.658228 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c42e16-86d7-491a-9381-07433058efee admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:54.824604 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca437fa7-fc55-4fee-bede-aae0712a159e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:54.825940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca437fa7-fc55-4fee-bede-aae0712a159e admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:54.826355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca437fa7-fc55-4fee-bede-aae0712a159e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:54.827124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca437fa7-fc55-4fee-bede-aae0712a159e admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:54.828018 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca437fa7-fc55-4fee-bede-aae0712a159e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:54.830413 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca437fa7-fc55-4fee-bede-aae0712a159e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:54.831612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 860/1714] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:54.834810 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8885812-c646-4ce4-9668-6e9be3aaa21c None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:54.836239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8885812-c646-4ce4-9668-6e9be3aaa21c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:54.836549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8885812-c646-4ce4-9668-6e9be3aaa21c None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:54.846102 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 861/1715] 10.222.0.44 () {54 vars in 960 bytes} [Tue Jul 12 22:05:54 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:05:54.848439 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8885812-c646-4ce4-9668-6e9be3aaa21c None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:54.880634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8885812-c646-4ce4-9668-6e9be3aaa21c None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:54.889541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 855/1716] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:54 2022] POST /identity/v3/users => generated 297 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:54.891821 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a994468-0c4a-4073-a317-4ae2f1161275 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:54.893974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a267d3-c700-4637-bee8-a4185199390a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:54.894717 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a267d3-c700-4637-bee8-a4185199390a admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:54.895004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a267d3-c700-4637-bee8-a4185199390a admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:54.895284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a267d3-c700-4637-bee8-a4185199390a admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:54.896031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a267d3-c700-4637-bee8-a4185199390a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:54.896531 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a994468-0c4a-4073-a317-4ae2f1161275 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:54.901645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a994468-0c4a-4073-a317-4ae2f1161275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:54.902584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a267d3-c700-4637-bee8-a4185199390a admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:54.927546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 862/1717] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:54.928418 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a267d3-c700-4637-bee8-a4185199390a admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:54.938095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-81a267d3-c700-4637-bee8-a4185199390a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:54.949928 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-6a994468-0c4a-4073-a317-4ae2f1161275 None None] MFA Rules not processed for user `a1f1ef52f8eb42a59a50a69873b9ffaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:54.954004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6a994468-0c4a-4073-a317-4ae2f1161275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:55.137942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a267d3-c700-4637-bee8-a4185199390a admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:55.167683 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 856/1718] 10.222.0.44 () {58 vars in 1057 bytes} [Tue Jul 12 22:05:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:55.169409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a994468-0c4a-4073-a317-4ae2f1161275 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:55.177911 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddd7a7-3c8c-4987-b8cb-1befc60d27ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:55.178932 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddd7a7-3c8c-4987-b8cb-1befc60d27ef service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:55.179316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddd7a7-3c8c-4987-b8cb-1befc60d27ef service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:55.180405 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddd7a7-3c8c-4987-b8cb-1befc60d27ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:55.195792 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a994468-0c4a-4073-a317-4ae2f1161275 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:55.203791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ddd7a7-3c8c-4987-b8cb-1befc60d27ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:55.206230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ddd7a7-3c8c-4987-b8cb-1befc60d27ef service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:55.212426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5c6b5-d76f-45ce-bf4d-7362e61cc968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:55.213276 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5c6b5-d76f-45ce-bf4d-7362e61cc968 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:55.213609 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5c6b5-d76f-45ce-bf4d-7362e61cc968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:55.213943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5c6b5-d76f-45ce-bf4d-7362e61cc968 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:55.215681 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5c6b5-d76f-45ce-bf4d-7362e61cc968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:55.219520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5c6b5-d76f-45ce-bf4d-7362e61cc968 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:55.226265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 863/1719] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:55.239168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddd7a7-3c8c-4987-b8cb-1befc60d27ef service nova] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:55.270893 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 857/1720] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:55.276798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973f707-afa8-442f-b26d-af01dcb77c0e None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:55.277136 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973f707-afa8-442f-b26d-af01dcb77c0e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:55.277424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973f707-afa8-442f-b26d-af01dcb77c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:55.293345 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddd7a7-3c8c-4987-b8cb-1befc60d27ef service nova] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:55.310469 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:55.311808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993be7f8-14c8-4f9b-97a9-620bd8918a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:55.314348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993be7f8-14c8-4f9b-97a9-620bd8918a4c admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:55.315384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993be7f8-14c8-4f9b-97a9-620bd8918a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:55.318369 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993be7f8-14c8-4f9b-97a9-620bd8918a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:55.319357 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993be7f8-14c8-4f9b-97a9-620bd8918a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:55.326652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993be7f8-14c8-4f9b-97a9-620bd8918a4c admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:55.375439 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 864/1721] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:55.376157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993be7f8-14c8-4f9b-97a9-620bd8918a4c admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:55.389211 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-993be7f8-14c8-4f9b-97a9-620bd8918a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:55.421199 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993be7f8-14c8-4f9b-97a9-620bd8918a4c admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:55.425438 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-1973f707-afa8-442f-b26d-af01dcb77c0e None None] MFA Rules not processed for user `4a746b42824c4d13a1d0b0f2c5831259`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:55.427186 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1973f707-afa8-442f-b26d-af01dcb77c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:55.437941 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:55.438843 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:55.439134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:55.439416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4 admin admin] PATH_INFO: `/v3/projects/23c82522098348f589785650ba590b55` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:55.440841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23c82522098348f589785650ba590b55)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:55.442814 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:55.518894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:55.580739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23c82522098348f589785650ba590b55', 'request_id': 'req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:55.595228 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23c82522098348f589785650ba590b55', 'request_id': 'req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:55.616630 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23c82522098348f589785650ba590b55', 'request_id': 'req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:55.641655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4 admin admin] The token cache is being invalidate because project 23c82522098348f589785650ba590b55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:05:55.641945 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:55.643387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:55.649320 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 865/1722] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:05:55 2022] DELETE /identity/v3/projects/23c82522098348f589785650ba590b55 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:55.650475 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:55.653797 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:55.688650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 858/1723] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:55.689807 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1973f707-afa8-442f-b26d-af01dcb77c0e None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:55.691713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1973f707-afa8-442f-b26d-af01dcb77c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:55.709031 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1973f707-afa8-442f-b26d-af01dcb77c0e None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:55.853642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aafd53-ebf5-40f3-9cdf-f06f8f897fa4 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:55.854974 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3c2dc-0644-44ff-b856-4f3c62b68e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:55.856122 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3c2dc-0644-44ff-b856-4f3c62b68e5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:55.859844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3c2dc-0644-44ff-b856-4f3c62b68e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:55.861676 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3c2dc-0644-44ff-b856-4f3c62b68e5d admin admin] PATH_INFO: `/v3/projects/e8db3288ce4743b49635435878722dfc/users/bef597abe146496698cce9c571228dc4/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:55.870636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03e234-15cf-4774-8c9b-8975478b6039 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f1ef52f8eb42a59a50a69873b9ffaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:55.871639 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03e234-15cf-4774-8c9b-8975478b6039 service placement] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:55.872018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03e234-15cf-4774-8c9b-8975478b6039 service placement] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:55.873809 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03e234-15cf-4774-8c9b-8975478b6039 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:55.880411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5c03e234-15cf-4774-8c9b-8975478b6039 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:55.911007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3c2dc-0644-44ff-b856-4f3c62b68e5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8db3288ce4743b49635435878722dfc, user_id=bef597abe146496698cce9c571228dc4, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:55.918037 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3c2dc-0644-44ff-b856-4f3c62b68e5d admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:55.922399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03e234-15cf-4774-8c9b-8975478b6039 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:55.925350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03e234-15cf-4774-8c9b-8975478b6039 service placement] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:55.951806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 859/1724] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:55 2022] PUT /identity/v3/projects/e8db3288ce4743b49635435878722dfc/users/bef597abe146496698cce9c571228dc4/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:55.957198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3c2dc-0644-44ff-b856-4f3c62b68e5d admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:55.962761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-43c3c2dc-0644-44ff-b856-4f3c62b68e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:55.963465 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 866/1725] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:55.967303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13040c42-8dff-4581-8df9-229867dec39f None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:55.968748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13040c42-8dff-4581-8df9-229867dec39f None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:55.969138 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13040c42-8dff-4581-8df9-229867dec39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:56.013037 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-13040c42-8dff-4581-8df9-229867dec39f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:56.014598 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-13040c42-8dff-4581-8df9-229867dec39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:56.119569 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3c2dc-0644-44ff-b856-4f3c62b68e5d admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:56.130381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cb1c1-a389-4df4-b49d-8798f35a9d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:56.137046 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cb1c1-a389-4df4-b49d-8798f35a9d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:56.137627 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cb1c1-a389-4df4-b49d-8798f35a9d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:56.138631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cb1c1-a389-4df4-b49d-8798f35a9d6d admin admin] PATH_INFO: `/v3/projects/8e7063e143a04ae98ab79f4e31ba525b/users/0c868c41878248ba9a934b50fdc1fa82/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:56.166673 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 867/1726] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:56.174029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13040c42-8dff-4581-8df9-229867dec39f None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:56.174029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-13040c42-8dff-4581-8df9-229867dec39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:56.178229 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873cb1c1-a389-4df4-b49d-8798f35a9d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e7063e143a04ae98ab79f4e31ba525b, user_id=0c868c41878248ba9a934b50fdc1fa82, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:56.186372 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873cb1c1-a389-4df4-b49d-8798f35a9d6d admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:56.205478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:56.231655 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 860/1727] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:55 2022] PUT /identity/v3/projects/8e7063e143a04ae98ab79f4e31ba525b/users/0c868c41878248ba9a934b50fdc1fa82/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:56.232768 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cb1c1-a389-4df4-b49d-8798f35a9d6d admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:56.287532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:56.398161 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13040c42-8dff-4581-8df9-229867dec39f None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:56.445895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cb1c1-a389-4df4-b49d-8798f35a9d6d admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:56.460187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2d86f-8b73-469b-a7e4-eb5dc8e89109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:56.461107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2d86f-8b73-469b-a7e4-eb5dc8e89109 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:56.462810 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2d86f-8b73-469b-a7e4-eb5dc8e89109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:56.463464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2d86f-8b73-469b-a7e4-eb5dc8e89109 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:56.468279 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2d86f-8b73-469b-a7e4-eb5dc8e89109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:56.474707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2d86f-8b73-469b-a7e4-eb5dc8e89109 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:56.514303 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 861/1728] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:56.518540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a9f5e-b7fc-4f77-9ad6-f054f8caa582 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:56.519378 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a9f5e-b7fc-4f77-9ad6-f054f8caa582 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:56.519880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a9f5e-b7fc-4f77-9ad6-f054f8caa582 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:56.527362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 862/1729] 10.222.0.44 () {54 vars in 921 bytes} [Tue Jul 12 22:05:56 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:05:56.530695 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121c3fe-8780-434b-ad0e-20bab49f6cdf None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:56.531139 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121c3fe-8780-434b-ad0e-20bab49f6cdf None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:56.533344 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121c3fe-8780-434b-ad0e-20bab49f6cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:56.596975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-a121c3fe-8780-434b-ad0e-20bab49f6cdf None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:56.603829 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a121c3fe-8780-434b-ad0e-20bab49f6cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:56.630917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922e716-cd81-4d03-9f9f-e9169e9ca4a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:56.631752 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 863/1730] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:56.633781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922e716-cd81-4d03-9f9f-e9169e9ca4a4 service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:56.634241 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922e716-cd81-4d03-9f9f-e9169e9ca4a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:56.634623 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922e716-cd81-4d03-9f9f-e9169e9ca4a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:56.635288 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ca0a9-7a68-41f7-95fc-f8f81325d907 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:56.635688 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ca0a9-7a68-41f7-95fc-f8f81325d907 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:56.636798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ca0a9-7a68-41f7-95fc-f8f81325d907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:56.709013 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922e716-cd81-4d03-9f9f-e9169e9ca4a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:56.710966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922e716-cd81-4d03-9f9f-e9169e9ca4a4 service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:56.722738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-6a4ca0a9-7a68-41f7-95fc-f8f81325d907 None None] MFA Rules not processed for user `0c868c41878248ba9a934b50fdc1fa82`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:56.723928 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 868/1731] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:56.725684 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922e716-cd81-4d03-9f9f-e9169e9ca4a4 service nova] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:56.727495 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d922e716-cd81-4d03-9f9f-e9169e9ca4a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:56.728590 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6a4ca0a9-7a68-41f7-95fc-f8f81325d907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:56.747840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922e716-cd81-4d03-9f9f-e9169e9ca4a4 service nova] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:56.762593 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af36e0-79a0-477a-b086-17df7138e37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:56.763965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af36e0-79a0-477a-b086-17df7138e37e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:56.764979 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af36e0-79a0-477a-b086-17df7138e37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:56.766229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af36e0-79a0-477a-b086-17df7138e37e admin admin] PATH_INFO: `/v3/users/21a02a34ae414ea3a3afbefe29709b74` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:56.773573 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af36e0-79a0-477a-b086-17df7138e37e admin admin] RBAC: Authorizing `identity:delete_user(user_id=21a02a34ae414ea3a3afbefe29709b74)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:56.778569 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af36e0-79a0-477a-b086-17df7138e37e admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:56.908309 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 864/1732] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:56.910343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ca0a9-7a68-41f7-95fc-f8f81325d907 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:56.920989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-42af36e0-79a0-477a-b086-17df7138e37e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21a02a34ae414ea3a3afbefe29709b74'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:56.942121 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ca0a9-7a68-41f7-95fc-f8f81325d907 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:56.962667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf588da3-1654-4ff1-9a23-8a50a5ad7d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:56.962667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf588da3-1654-4ff1-9a23-8a50a5ad7d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:56.962667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf588da3-1654-4ff1-9a23-8a50a5ad7d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:56.963024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf588da3-1654-4ff1-9a23-8a50a5ad7d7a admin admin] PATH_INFO: `/v3/projects/e8db3288ce4743b49635435878722dfc/users/bef597abe146496698cce9c571228dc4/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:56.968164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf588da3-1654-4ff1-9a23-8a50a5ad7d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8db3288ce4743b49635435878722dfc, user_id=bef597abe146496698cce9c571228dc4, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:56.975291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf588da3-1654-4ff1-9a23-8a50a5ad7d7a admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:56.991471 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-42af36e0-79a0-477a-b086-17df7138e37e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21a02a34ae414ea3a3afbefe29709b74'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:57.008309 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:57.020784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 865/1733] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:56 2022] PUT /identity/v3/projects/e8db3288ce4743b49635435878722dfc/users/bef597abe146496698cce9c571228dc4/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:57.020784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24102466-a7fd-44e7-984c-d83000d4914d None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:57.020784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24102466-a7fd-44e7-984c-d83000d4914d None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:57.020784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24102466-a7fd-44e7-984c-d83000d4914d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:57.037372 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-42af36e0-79a0-477a-b086-17df7138e37e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21a02a34ae414ea3a3afbefe29709b74'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:57.074078 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-42af36e0-79a0-477a-b086-17df7138e37e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21a02a34ae414ea3a3afbefe29709b74'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:57.077217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-42af36e0-79a0-477a-b086-17df7138e37e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21a02a34ae414ea3a3afbefe29709b74'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:57.090851 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 869/1734] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:05:56 2022] DELETE /identity/v3/users/21a02a34ae414ea3a3afbefe29709b74 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:57.092016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af36e0-79a0-477a-b086-17df7138e37e admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:57.097073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-24102466-a7fd-44e7-984c-d83000d4914d None None] MFA Rules not processed for user `a1f1ef52f8eb42a59a50a69873b9ffaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:57.098101 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-42af36e0-79a0-477a-b086-17df7138e37e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:57.100099 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-24102466-a7fd-44e7-984c-d83000d4914d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:57.276761 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af36e0-79a0-477a-b086-17df7138e37e admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:57.295516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3eb5d3-ba6c-40e9-9fd2-05c04c44320d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:57.296250 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3eb5d3-ba6c-40e9-9fd2-05c04c44320d admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:57.299103 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3eb5d3-ba6c-40e9-9fd2-05c04c44320d admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:57.300318 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3eb5d3-ba6c-40e9-9fd2-05c04c44320d admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:57.304737 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3eb5d3-ba6c-40e9-9fd2-05c04c44320d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:05:57.305178 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3eb5d3-ba6c-40e9-9fd2-05c04c44320d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:57.312867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3eb5d3-ba6c-40e9-9fd2-05c04c44320d admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:57.329533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:57.357647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 870/1735] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:05:57.357868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 866/1736] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Jul 12 22:05:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:57.357984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3eb5d3-ba6c-40e9-9fd2-05c04c44320d admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:57.358072 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bc7c8-d903-4684-8b7f-af7de91e8e7b None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:57.358072 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bc7c8-d903-4684-8b7f-af7de91e8e7b None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:57.358257 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bc7c8-d903-4684-8b7f-af7de91e8e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:57.361922 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ea3eb5d3-ba6c-40e9-9fd2-05c04c44320d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:57.400561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3eb5d3-ba6c-40e9-9fd2-05c04c44320d admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:57.417001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b3e49-f18b-4c44-ab1a-df71e115c175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:57.418620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b3e49-f18b-4c44-ab1a-df71e115c175 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:57.422510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b3e49-f18b-4c44-ab1a-df71e115c175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:57.424965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b3e49-f18b-4c44-ab1a-df71e115c175 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:57.433003 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b3e49-f18b-4c44-ab1a-df71e115c175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:57.443393 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b3e49-f18b-4c44-ab1a-df71e115c175 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:57.446796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-8f1bc7c8-d903-4684-8b7f-af7de91e8e7b None None] MFA Rules not processed for user `bef597abe146496698cce9c571228dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:57.453346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8f1bc7c8-d903-4684-8b7f-af7de91e8e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:57.522403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 871/1737] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:57 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:57.528325 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b3e49-f18b-4c44-ab1a-df71e115c175 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:57.654087 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 867/1738] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:57.655113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bc7c8-d903-4684-8b7f-af7de91e8e7b None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:57.667378 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8f1bc7c8-d903-4684-8b7f-af7de91e8e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:57.720406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bc7c8-d903-4684-8b7f-af7de91e8e7b None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:57.793256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b3e49-f18b-4c44-ab1a-df71e115c175 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:57.828846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fcc3d-9c8a-4191-b901-4e5cb00817cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:57.829930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fcc3d-9c8a-4191-b901-4e5cb00817cd admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:57.832066 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fcc3d-9c8a-4191-b901-4e5cb00817cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:57.832066 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fcc3d-9c8a-4191-b901-4e5cb00817cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:57.833216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85fcc3d-9c8a-4191-b901-4e5cb00817cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:57.841475 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85fcc3d-9c8a-4191-b901-4e5cb00817cd admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:57.915873 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89889659-ee8f-40b2-a5ac-e8239210a13e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f1ef52f8eb42a59a50a69873b9ffaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:57.916986 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89889659-ee8f-40b2-a5ac-e8239210a13e service placement] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:57.917709 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 872/1739] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:05:57 2022] POST /identity/v3/projects => generated 375 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:57.921695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70d165-02b6-43f9-b125-66b4ceb50a4c None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:57.922925 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70d165-02b6-43f9-b125-66b4ceb50a4c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:57.924262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70d165-02b6-43f9-b125-66b4ceb50a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:57.930055 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89889659-ee8f-40b2-a5ac-e8239210a13e service placement] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:57.932873 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89889659-ee8f-40b2-a5ac-e8239210a13e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:57.975716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89889659-ee8f-40b2-a5ac-e8239210a13e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:57.980627 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89889659-ee8f-40b2-a5ac-e8239210a13e service placement] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:58.001818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-6b70d165-02b6-43f9-b125-66b4ceb50a4c None None] MFA Rules not processed for user `bf13eeff770b406796ab7008b8429637`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:58.005773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-6b70d165-02b6-43f9-b125-66b4ceb50a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:58.021733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 868/1740] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:58.023542 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89889659-ee8f-40b2-a5ac-e8239210a13e service placement] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:58.035917 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89889659-ee8f-40b2-a5ac-e8239210a13e service placement] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:58.112679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:58.127975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3d07f-c515-42e0-877a-089dc8c97547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:58.128841 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3d07f-c515-42e0-877a-089dc8c97547 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:58.129198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3d07f-c515-42e0-877a-089dc8c97547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:58.129556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3d07f-c515-42e0-877a-089dc8c97547 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:58.130316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d3d07f-c515-42e0-877a-089dc8c97547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:58.133317 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d3d07f-c515-42e0-877a-089dc8c97547 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:58.167819 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 869/1741] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:58 2022] POST /identity/v3/users => generated 299 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:58.167819 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3d07f-c515-42e0-877a-089dc8c97547 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:58.168494 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c5d3d07f-c515-42e0-877a-089dc8c97547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:58.178310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:58.193810 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 873/1742] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:58.207032 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70d165-02b6-43f9-b125-66b4ceb50a4c None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:58.219301 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70d165-02b6-43f9-b125-66b4ceb50a4c None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:58.225611 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88a56b-d5b0-411d-b0e5-7a661bf5d730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:58.226003 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a56b-d5b0-411d-b0e5-7a661bf5d730 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:58.226167 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a56b-d5b0-411d-b0e5-7a661bf5d730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:58.226433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a56b-d5b0-411d-b0e5-7a661bf5d730 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:58.227112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88a56b-d5b0-411d-b0e5-7a661bf5d730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:58.230578 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88a56b-d5b0-411d-b0e5-7a661bf5d730 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:58.262484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 874/1743] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:05:58 2022] POST /identity/v3/users => generated 288 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:05:58.262484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88a56b-d5b0-411d-b0e5-7a661bf5d730 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:58.262705 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-7d88a56b-d5b0-411d-b0e5-7a661bf5d730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:58.276594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88a56b-d5b0-411d-b0e5-7a661bf5d730 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:58.282110 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c007b-a817-44dd-924d-890169eb6b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:58.283655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c007b-a817-44dd-924d-890169eb6b72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:58.283951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c007b-a817-44dd-924d-890169eb6b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:58.284269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c007b-a817-44dd-924d-890169eb6b72 admin admin] PATH_INFO: `/v3/projects/2bfddae441b9434eb18589926f360876` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:58.286418 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c007b-a817-44dd-924d-890169eb6b72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bfddae441b9434eb18589926f360876)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:58.289164 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c007b-a817-44dd-924d-890169eb6b72 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:58.326616 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3d07f-c515-42e0-877a-089dc8c97547 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:58.341807 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf9e71-8d68-4f4e-bece-6913b110afd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:58.344089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf9e71-8d68-4f4e-bece-6913b110afd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:58.344426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf9e71-8d68-4f4e-bece-6913b110afd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:58.346380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf9e71-8d68-4f4e-bece-6913b110afd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:58.377210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-870c007b-a817-44dd-924d-890169eb6b72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bfddae441b9434eb18589926f360876', 'request_id': 'req-870c007b-a817-44dd-924d-890169eb6b72'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:58.393651 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-870c007b-a817-44dd-924d-890169eb6b72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bfddae441b9434eb18589926f360876', 'request_id': 'req-870c007b-a817-44dd-924d-890169eb6b72'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:58.409763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-870c007b-a817-44dd-924d-890169eb6b72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bfddae441b9434eb18589926f360876', 'request_id': 'req-870c007b-a817-44dd-924d-890169eb6b72'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:58.438175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-870c007b-a817-44dd-924d-890169eb6b72 admin admin] The token cache is being invalidate because project 2bfddae441b9434eb18589926f360876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:05:58.438527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-870c007b-a817-44dd-924d-890169eb6b72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:58.440934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-870c007b-a817-44dd-924d-890169eb6b72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:58.447866 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 875/1744] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:05:58 2022] DELETE /identity/v3/projects/2bfddae441b9434eb18589926f360876 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:58.448916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c007b-a817-44dd-924d-890169eb6b72 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:58.451793 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-870c007b-a817-44dd-924d-890169eb6b72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:58.503162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf9e71-8d68-4f4e-bece-6913b110afd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:58.505049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf9e71-8d68-4f4e-bece-6913b110afd9 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:58.578221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 870/1745] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:58.580118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c22e2-46cf-46c1-991c-7d2fc80eb941 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:58.592727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c22e2-46cf-46c1-991c-7d2fc80eb941 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:58.592727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c22e2-46cf-46c1-991c-7d2fc80eb941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:58.612887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c007b-a817-44dd-924d-890169eb6b72 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:58.627222 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7214fb2-c779-4f36-b529-5a3e9403d877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:58.627997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7214fb2-c779-4f36-b529-5a3e9403d877 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:58.628351 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7214fb2-c779-4f36-b529-5a3e9403d877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:58.629478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7214fb2-c779-4f36-b529-5a3e9403d877 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:58.630335 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7214fb2-c779-4f36-b529-5a3e9403d877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:58.632090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7214fb2-c779-4f36-b529-5a3e9403d877 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:58.646537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 876/1746] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:58.647745 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7214fb2-c779-4f36-b529-5a3e9403d877 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:58.659235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e7214fb2-c779-4f36-b529-5a3e9403d877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:58.659828 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-4d5c22e2-46cf-46c1-991c-7d2fc80eb941 None None] MFA Rules not processed for user `bf13eeff770b406796ab7008b8429637`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:58.661107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4d5c22e2-46cf-46c1-991c-7d2fc80eb941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:58.678911 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7214fb2-c779-4f36-b529-5a3e9403d877 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:58.689120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd2e17-379a-4253-9670-b1603c369c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:58.690137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd2e17-379a-4253-9670-b1603c369c7a admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:58.690603 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd2e17-379a-4253-9670-b1603c369c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:58.691057 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd2e17-379a-4253-9670-b1603c369c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:58.692174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd2e17-379a-4253-9670-b1603c369c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:58.694125 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd2e17-379a-4253-9670-b1603c369c7a admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:58.766591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 877/1747] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:58.767894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd2e17-379a-4253-9670-b1603c369c7a admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:58.780764 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c3dd2e17-379a-4253-9670-b1603c369c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:58.827303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd2e17-379a-4253-9670-b1603c369c7a admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:58.848250 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438329c7-31f8-4229-b8c3-ff59e3ce86d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:58.849371 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438329c7-31f8-4229-b8c3-ff59e3ce86d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:58.857888 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438329c7-31f8-4229-b8c3-ff59e3ce86d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:58.858231 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438329c7-31f8-4229-b8c3-ff59e3ce86d0 admin admin] PATH_INFO: `/v3/users/9cdeaf848c1f4fd8b110f2ceeadd5b62` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:58.859921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438329c7-31f8-4229-b8c3-ff59e3ce86d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cdeaf848c1f4fd8b110f2ceeadd5b62)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:58.865192 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438329c7-31f8-4229-b8c3-ff59e3ce86d0 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:58.874798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 871/1748] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:05:58.874798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c22e2-46cf-46c1-991c-7d2fc80eb941 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:58.901557 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c22e2-46cf-46c1-991c-7d2fc80eb941 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:58.914802 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4f7b8-a28c-4ec6-b092-b6019cc09e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:58.916488 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4f7b8-a28c-4ec6-b092-b6019cc09e89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:58.917641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4f7b8-a28c-4ec6-b092-b6019cc09e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:58.917947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4f7b8-a28c-4ec6-b092-b6019cc09e89 admin admin] PATH_INFO: `/v3/projects/f047bfce250d45e9b0a872cfdeb426a0/users/19820303a0a8459e88f8ec8069da1e6f/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:58.933535 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:58.960301 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4f7b8-a28c-4ec6-b092-b6019cc09e89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f047bfce250d45e9b0a872cfdeb426a0, user_id=19820303a0a8459e88f8ec8069da1e6f, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:58.973143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4f7b8-a28c-4ec6-b092-b6019cc09e89 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:59.000205 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 872/1749] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:58 2022] PUT /identity/v3/projects/f047bfce250d45e9b0a872cfdeb426a0/users/19820303a0a8459e88f8ec8069da1e6f/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:59.001950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4f7b8-a28c-4ec6-b092-b6019cc09e89 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:59.007171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-438329c7-31f8-4229-b8c3-ff59e3ce86d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cdeaf848c1f4fd8b110f2ceeadd5b62'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:59.036262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-438329c7-31f8-4229-b8c3-ff59e3ce86d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cdeaf848c1f4fd8b110f2ceeadd5b62'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:59.046587 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-438329c7-31f8-4229-b8c3-ff59e3ce86d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cdeaf848c1f4fd8b110f2ceeadd5b62'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:59.061640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-438329c7-31f8-4229-b8c3-ff59e3ce86d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cdeaf848c1f4fd8b110f2ceeadd5b62'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:59.062829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-438329c7-31f8-4229-b8c3-ff59e3ce86d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cdeaf848c1f4fd8b110f2ceeadd5b62'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:05:59.082170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 878/1750] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:05:58 2022] DELETE /identity/v3/users/9cdeaf848c1f4fd8b110f2ceeadd5b62 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:59.084990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438329c7-31f8-4229-b8c3-ff59e3ce86d0 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:59.090815 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-438329c7-31f8-4229-b8c3-ff59e3ce86d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:59.115005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:59.168091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4f7b8-a28c-4ec6-b092-b6019cc09e89 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:59.297119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438329c7-31f8-4229-b8c3-ff59e3ce86d0 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:59.310510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032eab9-2f07-4b4b-810e-f936c8118dcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:59.311197 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032eab9-2f07-4b4b-810e-f936c8118dcc service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:59.311485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032eab9-2f07-4b4b-810e-f936c8118dcc service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:59.311871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032eab9-2f07-4b4b-810e-f936c8118dcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:59.316420 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b032eab9-2f07-4b4b-810e-f936c8118dcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:59.317004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38b984-d886-4949-a2c9-02245bf59eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:59.317703 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38b984-d886-4949-a2c9-02245bf59eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:59.317988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38b984-d886-4949-a2c9-02245bf59eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:59.318245 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38b984-d886-4949-a2c9-02245bf59eb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:59.319048 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38b984-d886-4949-a2c9-02245bf59eb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:59.321410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38b984-d886-4949-a2c9-02245bf59eb3 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:59.350068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 873/1751] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:59.350965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38b984-d886-4949-a2c9-02245bf59eb3 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:59.354130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ee38b984-d886-4949-a2c9-02245bf59eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:59.385324 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38b984-d886-4949-a2c9-02245bf59eb3 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:59.401424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd34c6f-8c26-407f-900b-d05f954d0a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:59.402117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd34c6f-8c26-407f-900b-d05f954d0a39 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:59.402586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd34c6f-8c26-407f-900b-d05f954d0a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:59.402873 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd34c6f-8c26-407f-900b-d05f954d0a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:59.403774 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd34c6f-8c26-407f-900b-d05f954d0a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:59.405638 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd34c6f-8c26-407f-900b-d05f954d0a39 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:59.433511 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 874/1752] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:05:59.434898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd34c6f-8c26-407f-900b-d05f954d0a39 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:59.437675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-2dd34c6f-8c26-407f-900b-d05f954d0a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:59.459604 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd34c6f-8c26-407f-900b-d05f954d0a39 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:59.485342 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471487a1-cefe-4a3a-a418-820af7c89d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:59.486272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471487a1-cefe-4a3a-a418-820af7c89d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:59.486547 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471487a1-cefe-4a3a-a418-820af7c89d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:59.486825 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471487a1-cefe-4a3a-a418-820af7c89d1e admin admin] PATH_INFO: `/v3/projects/ef2df8c0c5be453880b88a8b5e65835d/users/9d4fb2af9f6a42a7a56195b9e6c0e1a0/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:59.495686 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032eab9-2f07-4b4b-810e-f936c8118dcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:59.497654 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032eab9-2f07-4b4b-810e-f936c8118dcc service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:59.519961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 879/1753] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:05:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:05:59.520739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032eab9-2f07-4b4b-810e-f936c8118dcc service nova] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:59.522264 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471487a1-cefe-4a3a-a418-820af7c89d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2df8c0c5be453880b88a8b5e65835d, user_id=9d4fb2af9f6a42a7a56195b9e6c0e1a0, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:59.526183 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471487a1-cefe-4a3a-a418-820af7c89d1e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:59.528988 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b032eab9-2f07-4b4b-810e-f936c8118dcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:59.553157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 875/1754] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:59 2022] PUT /identity/v3/projects/ef2df8c0c5be453880b88a8b5e65835d/users/9d4fb2af9f6a42a7a56195b9e6c0e1a0/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:59.555009 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471487a1-cefe-4a3a-a418-820af7c89d1e admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:05:59.605304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:59.667178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471487a1-cefe-4a3a-a418-820af7c89d1e admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:59.671667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032eab9-2f07-4b4b-810e-f936c8118dcc service nova] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:05:59.673238 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7961557-9700-4a90-8804-f2ee58c458fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:59.674184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7961557-9700-4a90-8804-f2ee58c458fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:59.674566 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7961557-9700-4a90-8804-f2ee58c458fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:59.674948 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7961557-9700-4a90-8804-f2ee58c458fc admin admin] PATH_INFO: `/v3/projects/f047bfce250d45e9b0a872cfdeb426a0/users/19820303a0a8459e88f8ec8069da1e6f/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:59.677320 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7961557-9700-4a90-8804-f2ee58c458fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f047bfce250d45e9b0a872cfdeb426a0, user_id=19820303a0a8459e88f8ec8069da1e6f, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:59.681620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7961557-9700-4a90-8804-f2ee58c458fc admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:59.700554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 876/1755] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:05:59 2022] PUT /identity/v3/projects/f047bfce250d45e9b0a872cfdeb426a0/users/19820303a0a8459e88f8ec8069da1e6f/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:05:59.702167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5b6f9-ab1a-4410-b482-1679128f0c54 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:59.702553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5b6f9-ab1a-4410-b482-1679128f0c54 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:59.702952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5b6f9-ab1a-4410-b482-1679128f0c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:59.787329 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-3bf5b6f9-ab1a-4410-b482-1679128f0c54 None None] MFA Rules not processed for user `9d4fb2af9f6a42a7a56195b9e6c0e1a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:05:59.794802 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-3bf5b6f9-ab1a-4410-b482-1679128f0c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:05:59.845815 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd4879-9ebd-4134-97f7-ffbb888fb0f7 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef597abe146496698cce9c571228dc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8db3288ce4743b49635435878722dfc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:05:59.846511 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd4879-9ebd-4134-97f7-ffbb888fb0f7 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:59.846806 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd4879-9ebd-4134-97f7-ffbb888fb0f7 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:59.847088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd4879-9ebd-4134-97f7-ffbb888fb0f7 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:59.847832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd4879-9ebd-4134-97f7-ffbb888fb0f7 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:05:59.848146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd4879-9ebd-4134-97f7-ffbb888fb0f7 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:05:59.849803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd4879-9ebd-4134-97f7-ffbb888fb0f7 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:05:59.851693 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:05:59.876392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 880/1756] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:05:59.893856 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b7804-bcd2-4f50-b2f5-b669d0789f57 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:05:59.894200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b7804-bcd2-4f50-b2f5-b669d0789f57 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:05:59.895144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b7804-bcd2-4f50-b2f5-b669d0789f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:05:59.995988 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-d53b7804-bcd2-4f50-b2f5-b669d0789f57 None None] MFA Rules not processed for user `19820303a0a8459e88f8ec8069da1e6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:00.009457 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 877/1757] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:00.011075 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5b6f9-ab1a-4410-b482-1679128f0c54 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:00.011320 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d53b7804-bcd2-4f50-b2f5-b669d0789f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:00.121800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5b6f9-ab1a-4410-b482-1679128f0c54 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:00.132015 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2747533-4f52-4786-aab2-39e84233a048 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef597abe146496698cce9c571228dc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8db3288ce4743b49635435878722dfc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:00.134229 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2747533-4f52-4786-aab2-39e84233a048 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:00.134749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2747533-4f52-4786-aab2-39e84233a048 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:00.135391 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2747533-4f52-4786-aab2-39e84233a048 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:00.136473 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2747533-4f52-4786-aab2-39e84233a048 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:00.138681 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2747533-4f52-4786-aab2-39e84233a048 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:00.153412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 881/1758] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:05:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:00.157310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b7804-bcd2-4f50-b2f5-b669d0789f57 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:00.164283 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d53b7804-bcd2-4f50-b2f5-b669d0789f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:00.199050 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 878/1759] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:00 2022] POST /identity/v3/projects => generated 355 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:00.199050 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2747533-4f52-4786-aab2-39e84233a048 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:00.199618 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d2747533-4f52-4786-aab2-39e84233a048 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:00.247782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:00.272031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b7804-bcd2-4f50-b2f5-b669d0789f57 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:00.321724 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2747533-4f52-4786-aab2-39e84233a048 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:00.329380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792416b-1811-4842-8d32-7bae7aefb2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:00.331265 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792416b-1811-4842-8d32-7bae7aefb2f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:00.331744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792416b-1811-4842-8d32-7bae7aefb2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:00.332169 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792416b-1811-4842-8d32-7bae7aefb2f5 admin admin] PATH_INFO: `/v3/projects/4dec303b5897499880c705ba996bd428` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:00.334327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792416b-1811-4842-8d32-7bae7aefb2f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dec303b5897499880c705ba996bd428)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:00.338063 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792416b-1811-4842-8d32-7bae7aefb2f5 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:00.405484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d855b-888c-444e-aa7a-51a1ac4542fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:00.406442 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d855b-888c-444e-aa7a-51a1ac4542fb service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:00.406863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d855b-888c-444e-aa7a-51a1ac4542fb service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:00.407456 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d855b-888c-444e-aa7a-51a1ac4542fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:00.429822 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c792416b-1811-4842-8d32-7bae7aefb2f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dec303b5897499880c705ba996bd428', 'request_id': 'req-c792416b-1811-4842-8d32-7bae7aefb2f5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:00.444263 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c792416b-1811-4842-8d32-7bae7aefb2f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dec303b5897499880c705ba996bd428', 'request_id': 'req-c792416b-1811-4842-8d32-7bae7aefb2f5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:00.446800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:00.479536 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c792416b-1811-4842-8d32-7bae7aefb2f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dec303b5897499880c705ba996bd428', 'request_id': 'req-c792416b-1811-4842-8d32-7bae7aefb2f5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:00.522579 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c792416b-1811-4842-8d32-7bae7aefb2f5 admin admin] The token cache is being invalidate because project 4dec303b5897499880c705ba996bd428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:00.523042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c792416b-1811-4842-8d32-7bae7aefb2f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:00.523930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c792416b-1811-4842-8d32-7bae7aefb2f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:00.530297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 879/1760] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:00 2022] DELETE /identity/v3/projects/4dec303b5897499880c705ba996bd428 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:00.531652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792416b-1811-4842-8d32-7bae7aefb2f5 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:00.533516 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c792416b-1811-4842-8d32-7bae7aefb2f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:00.579136 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d855b-888c-444e-aa7a-51a1ac4542fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:00.581042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d855b-888c-444e-aa7a-51a1ac4542fb service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:00.594731 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 882/1761] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:00.595756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d855b-888c-444e-aa7a-51a1ac4542fb service nova] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:00.663344 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792416b-1811-4842-8d32-7bae7aefb2f5 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:00.675534 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0bb41-c4f3-4de4-94c4-c9ea13517f35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:00.675534 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0bb41-c4f3-4de4-94c4-c9ea13517f35 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:00.675534 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0bb41-c4f3-4de4-94c4-c9ea13517f35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:00.675534 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0bb41-c4f3-4de4-94c4-c9ea13517f35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:00.681572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-76a0bb41-c4f3-4de4-94c4-c9ea13517f35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:00.769190 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d855b-888c-444e-aa7a-51a1ac4542fb service nova] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:00.816517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0bb41-c4f3-4de4-94c4-c9ea13517f35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:00.818500 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0bb41-c4f3-4de4-94c4-c9ea13517f35 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:00.838968 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 880/1762] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:00.863006 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63773a22-70db-4584-bbf3-72829c842abe None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:00.863864 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63773a22-70db-4584-bbf3-72829c842abe None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:00.864158 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63773a22-70db-4584-bbf3-72829c842abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:00.900187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-63773a22-70db-4584-bbf3-72829c842abe None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:00.901538 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-63773a22-70db-4584-bbf3-72829c842abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:00.933408 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:00.988025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82135b4-c76c-488c-9258-72d22e8b6492 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef597abe146496698cce9c571228dc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8db3288ce4743b49635435878722dfc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:00.988884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82135b4-c76c-488c-9258-72d22e8b6492 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:00.989181 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82135b4-c76c-488c-9258-72d22e8b6492 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:00.989465 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82135b4-c76c-488c-9258-72d22e8b6492 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:00.995262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82135b4-c76c-488c-9258-72d22e8b6492 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:00.995662 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82135b4-c76c-488c-9258-72d22e8b6492 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:00.997812 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82135b4-c76c-488c-9258-72d22e8b6492 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:01.018313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 883/1763] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:06:01.018313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9235f52-92bd-4e55-92c2-2fcf98fe6486 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:01.018313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9235f52-92bd-4e55-92c2-2fcf98fe6486 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:01.018313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9235f52-92bd-4e55-92c2-2fcf98fe6486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:01.059329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-c9235f52-92bd-4e55-92c2-2fcf98fe6486 None None] MFA Rules not processed for user `85dea53a4a034f5981b638bf92f2cf3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:01.064422 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c9235f52-92bd-4e55-92c2-2fcf98fe6486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:01.079898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 881/1764] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:01.081400 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733c5ce-c51b-4589-a5a5-1d46f079f45c None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:01.081692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733c5ce-c51b-4589-a5a5-1d46f079f45c None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:01.081974 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733c5ce-c51b-4589-a5a5-1d46f079f45c None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:01.083414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 882/1765] 10.222.0.44 () {54 vars in 912 bytes} [Tue Jul 12 22:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:06:01.088670 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733c5ce-c51b-4589-a5a5-1d46f079f45c None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:01.093148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c733c5ce-c51b-4589-a5a5-1d46f079f45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:01.137667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733c5ce-c51b-4589-a5a5-1d46f079f45c None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:01.149359 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2cd41-05b7-4a9d-9c8f-f2f31593dd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:01.150595 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2cd41-05b7-4a9d-9c8f-f2f31593dd01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:01.150877 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2cd41-05b7-4a9d-9c8f-f2f31593dd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:01.151154 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2cd41-05b7-4a9d-9c8f-f2f31593dd01 admin admin] PATH_INFO: `/v3/users/9d4fb2af9f6a42a7a56195b9e6c0e1a0` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:01.153690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2cd41-05b7-4a9d-9c8f-f2f31593dd01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d4fb2af9f6a42a7a56195b9e6c0e1a0)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:01.155643 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2cd41-05b7-4a9d-9c8f-f2f31593dd01 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:01.184324 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 884/1766] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:01.186321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca871238-1ab3-4fa1-ae27-2cdae3eb08a9 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:01.186723 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca871238-1ab3-4fa1-ae27-2cdae3eb08a9 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:01.187043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca871238-1ab3-4fa1-ae27-2cdae3eb08a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:01.217658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-ca871238-1ab3-4fa1-ae27-2cdae3eb08a9 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:01.219058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ca871238-1ab3-4fa1-ae27-2cdae3eb08a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:01.235197 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 885/1767] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:01.236917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774b3ac-d8d9-4a72-8076-27c3c43ef288 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:01.237289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774b3ac-d8d9-4a72-8076-27c3c43ef288 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:01.237571 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774b3ac-d8d9-4a72-8076-27c3c43ef288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:01.240173 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:INSERT=1,keystone:DELETE=8,keystone:SELECT=91 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:01.283774 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-19e2cd41-05b7-4a9d-9c8f-f2f31593dd01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d4fb2af9f6a42a7a56195b9e6c0e1a0'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:01.298069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-19e2cd41-05b7-4a9d-9c8f-f2f31593dd01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d4fb2af9f6a42a7a56195b9e6c0e1a0'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:01.299269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-e774b3ac-d8d9-4a72-8076-27c3c43ef288 None None] MFA Rules not processed for user `53b891a734a64b5983564e207de0359d`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:01.300739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e774b3ac-d8d9-4a72-8076-27c3c43ef288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:01.318284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-19e2cd41-05b7-4a9d-9c8f-f2f31593dd01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d4fb2af9f6a42a7a56195b9e6c0e1a0'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:01.318943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-19e2cd41-05b7-4a9d-9c8f-f2f31593dd01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d4fb2af9f6a42a7a56195b9e6c0e1a0'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:01.319884 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-19e2cd41-05b7-4a9d-9c8f-f2f31593dd01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d4fb2af9f6a42a7a56195b9e6c0e1a0'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:01.358775 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 883/1768] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:01 2022] DELETE /identity/v3/users/9d4fb2af9f6a42a7a56195b9e6c0e1a0 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:01.359665 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2cd41-05b7-4a9d-9c8f-f2f31593dd01 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:01.367315 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-19e2cd41-05b7-4a9d-9c8f-f2f31593dd01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:01.419432 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 886/1769] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Jul 12 22:06:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:01.420453 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774b3ac-d8d9-4a72-8076-27c3c43ef288 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:01.423831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e774b3ac-d8d9-4a72-8076-27c3c43ef288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:01.449586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774b3ac-d8d9-4a72-8076-27c3c43ef288 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:01.530518 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2cd41-05b7-4a9d-9c8f-f2f31593dd01 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:01.543753 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542da0a3-813d-498f-a913-b295ecb87445 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef597abe146496698cce9c571228dc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e8db3288ce4743b49635435878722dfc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:01.544561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542da0a3-813d-498f-a913-b295ecb87445 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:01.544889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542da0a3-813d-498f-a913-b295ecb87445 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:01.545168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542da0a3-813d-498f-a913-b295ecb87445 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] PATH_INFO: `/v3/projects/d05046da2812400e9e7e4e814937c374` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:01.553262 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542da0a3-813d-498f-a913-b295ecb87445 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d05046da2812400e9e7e4e814937c374)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:01.553262 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542da0a3-813d-498f-a913-b295ecb87445 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:01.556772 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e0ae6-75bc-419b-b4d6-c894850d3ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:01.557760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e0ae6-75bc-419b-b4d6-c894850d3ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:01.559308 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e0ae6-75bc-419b-b4d6-c894850d3ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:01.559600 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e0ae6-75bc-419b-b4d6-c894850d3ac5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:01.560362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e0ae6-75bc-419b-b4d6-c894850d3ac5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:01.560670 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e0ae6-75bc-419b-b4d6-c894850d3ac5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:01.569809 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e0ae6-75bc-419b-b4d6-c894850d3ac5 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:01.599114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 887/1770] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:06:01.600681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cf81c-2594-4246-8ef1-634ec1f56d07 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:01.601033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cf81c-2594-4246-8ef1-634ec1f56d07 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:01.601317 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cf81c-2594-4246-8ef1-634ec1f56d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:01.618763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:01.649963 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-542da0a3-813d-498f-a913-b295ecb87445 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd05046da2812400e9e7e4e814937c374', 'request_id': 'req-542da0a3-813d-498f-a913-b295ecb87445'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:01.650898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-8f8cf81c-2594-4246-8ef1-634ec1f56d07 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:01.652347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8f8cf81c-2594-4246-8ef1-634ec1f56d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:01.660757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-542da0a3-813d-498f-a913-b295ecb87445 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd05046da2812400e9e7e4e814937c374', 'request_id': 'req-542da0a3-813d-498f-a913-b295ecb87445'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:01.675405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-542da0a3-813d-498f-a913-b295ecb87445 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd05046da2812400e9e7e4e814937c374', 'request_id': 'req-542da0a3-813d-498f-a913-b295ecb87445'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:01.717752 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-542da0a3-813d-498f-a913-b295ecb87445 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] The token cache is being invalidate because project d05046da2812400e9e7e4e814937c374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:01.718875 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-542da0a3-813d-498f-a913-b295ecb87445 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:01.724822 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-542da0a3-813d-498f-a913-b295ecb87445 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:01.731765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 884/1771] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:01 2022] DELETE /identity/v3/projects/d05046da2812400e9e7e4e814937c374 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:01.733373 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542da0a3-813d-498f-a913-b295ecb87445 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:01.738189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-542da0a3-813d-498f-a913-b295ecb87445 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:01.819245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 888/1772] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:01.826308 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cf81c-2594-4246-8ef1-634ec1f56d07 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:01.828635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8f8cf81c-2594-4246-8ef1-634ec1f56d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:01.857803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cf81c-2594-4246-8ef1-634ec1f56d07 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:01.867362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc64f7-f0b2-4e8b-97e7-4686b0b4d2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:01.868525 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc64f7-f0b2-4e8b-97e7-4686b0b4d2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:01.869079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc64f7-f0b2-4e8b-97e7-4686b0b4d2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:01.869645 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc64f7-f0b2-4e8b-97e7-4686b0b4d2f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:01.870838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc64f7-f0b2-4e8b-97e7-4686b0b4d2f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:01.874798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc64f7-f0b2-4e8b-97e7-4686b0b4d2f6 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:01.891233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:01.935236 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 889/1773] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:01 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:01.939959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc64f7-f0b2-4e8b-97e7-4686b0b4d2f6 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:01.947384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-07dc64f7-f0b2-4e8b-97e7-4686b0b4d2f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:01.978894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542da0a3-813d-498f-a913-b295ecb87445 tempest-VolumeQuotaClassesTest-1978866226 tempest-VolumeQuotaClassesTest-1978866226-project-admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:02.158246 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc64f7-f0b2-4e8b-97e7-4686b0b4d2f6 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:02.162960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db367f6-b42b-4050-8a43-5190c8a5569e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:02.168560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db367f6-b42b-4050-8a43-5190c8a5569e service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:02.168911 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db367f6-b42b-4050-8a43-5190c8a5569e service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:02.170059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bea55-70e6-42e8-88d5-dd8ea143d88a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:02.172118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db367f6-b42b-4050-8a43-5190c8a5569e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:02.172564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bea55-70e6-42e8-88d5-dd8ea143d88a service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:02.172896 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bea55-70e6-42e8-88d5-dd8ea143d88a service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:02.173195 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bea55-70e6-42e8-88d5-dd8ea143d88a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:02.176171 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0db367f6-b42b-4050-8a43-5190c8a5569e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:02.211789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db367f6-b42b-4050-8a43-5190c8a5569e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:02.214525 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db367f6-b42b-4050-8a43-5190c8a5569e service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:02.271406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 885/1774] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:02.272411 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db367f6-b42b-4050-8a43-5190c8a5569e service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:02.308092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bea55-70e6-42e8-88d5-dd8ea143d88a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:02.311532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bea55-70e6-42e8-88d5-dd8ea143d88a service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:02.338885 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 890/1775] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:02.341103 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3dadb-4ac6-4941-885d-9cf6e19ea711 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:02.341426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3dadb-4ac6-4941-885d-9cf6e19ea711 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:02.341881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3dadb-4ac6-4941-885d-9cf6e19ea711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:02.388840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-b9b3dadb-4ac6-4941-885d-9cf6e19ea711 None None] MFA Rules not processed for user `4a746b42824c4d13a1d0b0f2c5831259`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:02.391834 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b9b3dadb-4ac6-4941-885d-9cf6e19ea711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:02.457457 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db367f6-b42b-4050-8a43-5190c8a5569e service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:02.465023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:02.469979 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270aa9f-eed6-4f2d-8840-ee29e80638d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:02.471035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270aa9f-eed6-4f2d-8840-ee29e80638d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:02.471496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270aa9f-eed6-4f2d-8840-ee29e80638d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:02.471983 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270aa9f-eed6-4f2d-8840-ee29e80638d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:02.472949 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270aa9f-eed6-4f2d-8840-ee29e80638d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:02.483081 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270aa9f-eed6-4f2d-8840-ee29e80638d8 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:02.551446 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 886/1776] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:02 2022] POST /identity/v3/users => generated 287 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:02.552213 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270aa9f-eed6-4f2d-8840-ee29e80638d8 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:02.556747 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-f270aa9f-eed6-4f2d-8840-ee29e80638d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:02.565763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 891/1777] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:02.567048 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0be4e-face-42c4-a62b-c75147f7d719 None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:02.569701 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0be4e-face-42c4-a62b-c75147f7d719 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:02.570019 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0be4e-face-42c4-a62b-c75147f7d719 None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:02.574619 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 892/1778] 10.222.0.44 () {54 vars in 912 bytes} [Tue Jul 12 22:06:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:06:02.575500 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0be4e-face-42c4-a62b-c75147f7d719 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:02.596210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270aa9f-eed6-4f2d-8840-ee29e80638d8 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:02.640263 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b82a0-105a-46f4-b3c9-44b4752a47e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:02.641219 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b82a0-105a-46f4-b3c9-44b4752a47e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:02.641499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b82a0-105a-46f4-b3c9-44b4752a47e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:02.643452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0be4e-face-42c4-a62b-c75147f7d719 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:02.648571 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b82a0-105a-46f4-b3c9-44b4752a47e1 admin admin] PATH_INFO: `/v3/users/23873b8219414fdfa5ce15f7e7aab99b` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:02.653379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b82a0-105a-46f4-b3c9-44b4752a47e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23873b8219414fdfa5ce15f7e7aab99b)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:02.657684 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b82a0-105a-46f4-b3c9-44b4752a47e1 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:02.658051 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678df7e-db67-4a58-ae46-27a959e2c1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:02.659000 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678df7e-db67-4a58-ae46-27a959e2c1cd admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:02.659290 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678df7e-db67-4a58-ae46-27a959e2c1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:02.659570 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678df7e-db67-4a58-ae46-27a959e2c1cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:02.660315 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678df7e-db67-4a58-ae46-27a959e2c1cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:02.661973 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678df7e-db67-4a58-ae46-27a959e2c1cd admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:02.725552 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 893/1779] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:02.734030 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4407041-8475-4e88-a7c4-4dbcd2e59c22 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:02.741692 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4407041-8475-4e88-a7c4-4dbcd2e59c22 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:02.742323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4407041-8475-4e88-a7c4-4dbcd2e59c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:02.793065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-b4407041-8475-4e88-a7c4-4dbcd2e59c22 None None] MFA Rules not processed for user `53b891a734a64b5983564e207de0359d`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:02.795555 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b4407041-8475-4e88-a7c4-4dbcd2e59c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:02.817621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 894/1780] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Jul 12 22:06:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:02.818827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4407041-8475-4e88-a7c4-4dbcd2e59c22 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:02.849120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4407041-8475-4e88-a7c4-4dbcd2e59c22 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:02.849656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-123b82a0-105a-46f4-b3c9-44b4752a47e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23873b8219414fdfa5ce15f7e7aab99b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:02.865800 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e945a-4ace-4ff5-b687-81dd2354bae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:02.866843 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e945a-4ace-4ff5-b687-81dd2354bae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:02.868912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e945a-4ace-4ff5-b687-81dd2354bae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:02.869445 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e945a-4ace-4ff5-b687-81dd2354bae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:02.872931 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e945a-4ace-4ff5-b687-81dd2354bae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:02.883924 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e945a-4ace-4ff5-b687-81dd2354bae6 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:02.888211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-123b82a0-105a-46f4-b3c9-44b4752a47e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23873b8219414fdfa5ce15f7e7aab99b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:02.917501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 895/1781] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:02.918826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-123b82a0-105a-46f4-b3c9-44b4752a47e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23873b8219414fdfa5ce15f7e7aab99b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:02.919744 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e945a-4ace-4ff5-b687-81dd2354bae6 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:02.922551 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-123b82a0-105a-46f4-b3c9-44b4752a47e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23873b8219414fdfa5ce15f7e7aab99b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:02.927059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-754e945a-4ace-4ff5-b687-81dd2354bae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:02.927237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-123b82a0-105a-46f4-b3c9-44b4752a47e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23873b8219414fdfa5ce15f7e7aab99b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:02.957661 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e945a-4ace-4ff5-b687-81dd2354bae6 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:02.970299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680a715-0d4b-4a0e-be2e-11e8064dcb93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:02.990208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680a715-0d4b-4a0e-be2e-11e8064dcb93 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:02.990208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680a715-0d4b-4a0e-be2e-11e8064dcb93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:02.990208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680a715-0d4b-4a0e-be2e-11e8064dcb93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:02.999828 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:03.005121 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-9680a715-0d4b-4a0e-be2e-11e8064dcb93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:03.036072 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 887/1782] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:02 2022] DELETE /identity/v3/users/23873b8219414fdfa5ce15f7e7aab99b => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:03.037742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b82a0-105a-46f4-b3c9-44b4752a47e1 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:03.041189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-123b82a0-105a-46f4-b3c9-44b4752a47e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:03.050482 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680a715-0d4b-4a0e-be2e-11e8064dcb93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:03.051969 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680a715-0d4b-4a0e-be2e-11e8064dcb93 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:03.082940 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 896/1783] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:03.091847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a12a4a-c020-46b6-8b54-db9b512bc010 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:03.092123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a12a4a-c020-46b6-8b54-db9b512bc010 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:03.092405 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a12a4a-c020-46b6-8b54-db9b512bc010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:03.166992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-25a12a4a-c020-46b6-8b54-db9b512bc010 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:03.168385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-25a12a4a-c020-46b6-8b54-db9b512bc010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:03.216276 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b82a0-105a-46f4-b3c9-44b4752a47e1 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:03.223652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b59c2-ed56-42a1-898b-06973680db7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:03.224838 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b59c2-ed56-42a1-898b-06973680db7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:03.225235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b59c2-ed56-42a1-898b-06973680db7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:03.225706 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b59c2-ed56-42a1-898b-06973680db7c admin admin] PATH_INFO: `/v3/projects/5e72ec106cd04d4b9d7e51f144426f52/users/31df89f3225f4f1db7a4699fe83cc2cf/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:03.273294 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b59c2-ed56-42a1-898b-06973680db7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e72ec106cd04d4b9d7e51f144426f52, user_id=31df89f3225f4f1db7a4699fe83cc2cf, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:03.284445 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b59c2-ed56-42a1-898b-06973680db7c admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:03.354636 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 888/1784] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:03 2022] PUT /identity/v3/projects/5e72ec106cd04d4b9d7e51f144426f52/users/31df89f3225f4f1db7a4699fe83cc2cf/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:03.368617 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c858768-e946-4c8e-8a29-3a76f9774c01 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:03.369274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c858768-e946-4c8e-8a29-3a76f9774c01 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:03.369905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c858768-e946-4c8e-8a29-3a76f9774c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:03.388295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 897/1785] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:03.398113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a12a4a-c020-46b6-8b54-db9b512bc010 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:03.401629 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-25a12a4a-c020-46b6-8b54-db9b512bc010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:03.464498 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-7c858768-e946-4c8e-8a29-3a76f9774c01 None None] MFA Rules not processed for user `31df89f3225f4f1db7a4699fe83cc2cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:03.465794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-7c858768-e946-4c8e-8a29-3a76f9774c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:03.526335 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a12a4a-c020-46b6-8b54-db9b512bc010 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:03.652864 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 889/1786] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:03.654022 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c858768-e946-4c8e-8a29-3a76f9774c01 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:03.655402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-7c858768-e946-4c8e-8a29-3a76f9774c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:03.679181 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e386bb-05c0-45b8-8f72-13333e21ee59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:03.679874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e386bb-05c0-45b8-8f72-13333e21ee59 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:03.680383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e386bb-05c0-45b8-8f72-13333e21ee59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:03.680675 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e386bb-05c0-45b8-8f72-13333e21ee59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:03.694050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:03.801733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c858768-e946-4c8e-8a29-3a76f9774c01 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:03.818077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7facba2-72f5-4f20-8f11-b1ea66e81aa7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f1ef52f8eb42a59a50a69873b9ffaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:03.818749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7facba2-72f5-4f20-8f11-b1ea66e81aa7 service placement] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:03.819033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7facba2-72f5-4f20-8f11-b1ea66e81aa7 service placement] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:03.819309 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7facba2-72f5-4f20-8f11-b1ea66e81aa7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:03.830140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c7facba2-72f5-4f20-8f11-b1ea66e81aa7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:03.840470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e386bb-05c0-45b8-8f72-13333e21ee59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:03.842162 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e386bb-05c0-45b8-8f72-13333e21ee59 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:03.865297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7facba2-72f5-4f20-8f11-b1ea66e81aa7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:03.871123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7facba2-72f5-4f20-8f11-b1ea66e81aa7 service placement] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:03.884361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 898/1787] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:03.885393 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e386bb-05c0-45b8-8f72-13333e21ee59 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:03.887884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-f7e386bb-05c0-45b8-8f72-13333e21ee59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:03.890071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:03.915539 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 890/1788] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:03.918778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7facba2-72f5-4f20-8f11-b1ea66e81aa7 service placement] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:03.951601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7facba2-72f5-4f20-8f11-b1ea66e81aa7 service placement] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:03.973582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01b0b9-8e0c-4a06-beb3-084ccf616c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:03.974305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01b0b9-8e0c-4a06-beb3-084ccf616c2b admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:03.976111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01b0b9-8e0c-4a06-beb3-084ccf616c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:03.976424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01b0b9-8e0c-4a06-beb3-084ccf616c2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:03.977297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd01b0b9-8e0c-4a06-beb3-084ccf616c2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:03.979315 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd01b0b9-8e0c-4a06-beb3-084ccf616c2b admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:04.060295 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 891/1789] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:03 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:04.060295 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01b0b9-8e0c-4a06-beb3-084ccf616c2b admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:04.061215 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-fd01b0b9-8e0c-4a06-beb3-084ccf616c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:04.062017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e386bb-05c0-45b8-8f72-13333e21ee59 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:04.205615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc1e02-6aec-49aa-a055-3d614642a2c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:04.207662 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc1e02-6aec-49aa-a055-3d614642a2c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:04.208014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc1e02-6aec-49aa-a055-3d614642a2c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:04.208301 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc1e02-6aec-49aa-a055-3d614642a2c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:04.215464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c8cc1e02-6aec-49aa-a055-3d614642a2c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:04.247935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01b0b9-8e0c-4a06-beb3-084ccf616c2b admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:04.264341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cb6f6-ef77-43cf-afb8-1ecd67ffb9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:04.265292 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cb6f6-ef77-43cf-afb8-1ecd67ffb9f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:04.265609 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cb6f6-ef77-43cf-afb8-1ecd67ffb9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:04.265920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cb6f6-ef77-43cf-afb8-1ecd67ffb9f8 admin admin] PATH_INFO: `/v3/projects/ef2df8c0c5be453880b88a8b5e65835d` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:04.267802 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cb6f6-ef77-43cf-afb8-1ecd67ffb9f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef2df8c0c5be453880b88a8b5e65835d)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:04.270033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cb6f6-ef77-43cf-afb8-1ecd67ffb9f8 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:04.409542 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2a0cb6f6-ef77-43cf-afb8-1ecd67ffb9f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef2df8c0c5be453880b88a8b5e65835d', 'request_id': 'req-2a0cb6f6-ef77-43cf-afb8-1ecd67ffb9f8'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:04.425857 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2a0cb6f6-ef77-43cf-afb8-1ecd67ffb9f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef2df8c0c5be453880b88a8b5e65835d', 'request_id': 'req-2a0cb6f6-ef77-43cf-afb8-1ecd67ffb9f8'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:04.449725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc1e02-6aec-49aa-a055-3d614642a2c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:04.451486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2a0cb6f6-ef77-43cf-afb8-1ecd67ffb9f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef2df8c0c5be453880b88a8b5e65835d', 'request_id': 'req-2a0cb6f6-ef77-43cf-afb8-1ecd67ffb9f8'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:04.472205 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc1e02-6aec-49aa-a055-3d614642a2c6 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:04.506729 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2a0cb6f6-ef77-43cf-afb8-1ecd67ffb9f8 admin admin] The token cache is being invalidate because project ef2df8c0c5be453880b88a8b5e65835d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:04.506729 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2a0cb6f6-ef77-43cf-afb8-1ecd67ffb9f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:04.513019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2a0cb6f6-ef77-43cf-afb8-1ecd67ffb9f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:04.522159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 899/1790] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:04.524034 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc1e02-6aec-49aa-a055-3d614642a2c6 service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:04.542743 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 892/1791] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:04 2022] DELETE /identity/v3/projects/ef2df8c0c5be453880b88a8b5e65835d => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:04.552443 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc9bfe-2b18-4560-9f9d-b1be84599128 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:04.552443 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc9bfe-2b18-4560-9f9d-b1be84599128 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:04.553227 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc9bfe-2b18-4560-9f9d-b1be84599128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:04.563511 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc1e02-6aec-49aa-a055-3d614642a2c6 service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:04.592846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:04.699987 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-e0bc9bfe-2b18-4560-9f9d-b1be84599128 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:04.701351 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e0bc9bfe-2b18-4560-9f9d-b1be84599128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:04.720759 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6158258-e0d6-4267-bc3d-93fb2beeb309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:04.724278 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6158258-e0d6-4267-bc3d-93fb2beeb309 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:04.727172 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6158258-e0d6-4267-bc3d-93fb2beeb309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:04.727789 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6158258-e0d6-4267-bc3d-93fb2beeb309 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:04.730021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6158258-e0d6-4267-bc3d-93fb2beeb309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:04.732746 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6158258-e0d6-4267-bc3d-93fb2beeb309 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:04.788830 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 900/1792] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:04 2022] POST /identity/v3/users => generated 293 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:04.793353 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6158258-e0d6-4267-bc3d-93fb2beeb309 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:04.809661 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c6158258-e0d6-4267-bc3d-93fb2beeb309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:04.842142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6158258-e0d6-4267-bc3d-93fb2beeb309 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:04.856679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b9d99-4b24-410b-9768-97624ba33609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:04.858124 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b9d99-4b24-410b-9768-97624ba33609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:04.858619 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b9d99-4b24-410b-9768-97624ba33609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:04.859701 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b9d99-4b24-410b-9768-97624ba33609 admin admin] PATH_INFO: `/v3/users/02158c23639543618aa319de5be4aadc` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:04.864331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b9d99-4b24-410b-9768-97624ba33609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02158c23639543618aa319de5be4aadc)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:04.867467 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b9d99-4b24-410b-9768-97624ba33609 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:04.947109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 893/1793] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:04.953966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc9bfe-2b18-4560-9f9d-b1be84599128 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:04.955165 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e0bc9bfe-2b18-4560-9f9d-b1be84599128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:04.983855 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc9bfe-2b18-4560-9f9d-b1be84599128 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:04.985491 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d90b9d99-4b24-410b-9768-97624ba33609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02158c23639543618aa319de5be4aadc'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:04.997524 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fcd29-ed9c-49a6-88b7-4667abe63879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:04.998274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fcd29-ed9c-49a6-88b7-4667abe63879 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:04.998561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fcd29-ed9c-49a6-88b7-4667abe63879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:04.998843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fcd29-ed9c-49a6-88b7-4667abe63879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:04.999721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fcd29-ed9c-49a6-88b7-4667abe63879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:05.001533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fcd29-ed9c-49a6-88b7-4667abe63879 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:05.003211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:05.030835 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 894/1794] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:05.031915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fcd29-ed9c-49a6-88b7-4667abe63879 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:05.037912 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ba5fcd29-ed9c-49a6-88b7-4667abe63879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:05.040173 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d90b9d99-4b24-410b-9768-97624ba33609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02158c23639543618aa319de5be4aadc'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:05.051402 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d90b9d99-4b24-410b-9768-97624ba33609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02158c23639543618aa319de5be4aadc'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:05.061499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fcd29-ed9c-49a6-88b7-4667abe63879 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:05.067496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:05.068820 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:05.069108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:05.069385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201 admin admin] PATH_INFO: `/v3/projects/4d6bcba1180b438ab9c25e01b3967728` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:05.071103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d6bcba1180b438ab9c25e01b3967728)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:05.073417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:05.084781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d90b9d99-4b24-410b-9768-97624ba33609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02158c23639543618aa319de5be4aadc'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:05.085641 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d90b9d99-4b24-410b-9768-97624ba33609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02158c23639543618aa319de5be4aadc'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:05.098313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 901/1795] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:04 2022] DELETE /identity/v3/users/02158c23639543618aa319de5be4aadc => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:05.107730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078218d8-6e6c-4b9c-b9c9-dd63123c11a8 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:05.109528 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078218d8-6e6c-4b9c-b9c9-dd63123c11a8 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:05.111037 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078218d8-6e6c-4b9c-b9c9-dd63123c11a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:05.135948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-078218d8-6e6c-4b9c-b9c9-dd63123c11a8 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:05.137926 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-078218d8-6e6c-4b9c-b9c9-dd63123c11a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:05.157484 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d6bcba1180b438ab9c25e01b3967728', 'request_id': 'req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:05.174401 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d6bcba1180b438ab9c25e01b3967728', 'request_id': 'req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:05.193958 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d6bcba1180b438ab9c25e01b3967728', 'request_id': 'req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:05.228235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201 admin admin] The token cache is being invalidate because project 4d6bcba1180b438ab9c25e01b3967728 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:05.228980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:05.231065 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:05.237255 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 895/1796] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:05 2022] DELETE /identity/v3/projects/4d6bcba1180b438ab9c25e01b3967728 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:05.238629 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:05.241969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:05.343627 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 902/1797] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:05.348937 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847a75f-f4e0-4e4c-a3ce-216843c6439b None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:05.349466 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847a75f-f4e0-4e4c-a3ce-216843c6439b None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:05.354856 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847a75f-f4e0-4e4c-a3ce-216843c6439b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:05.411099 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc6f4b-8979-4b23-9f14-7b9cd02f1201 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:05.419026 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-8847a75f-f4e0-4e4c-a3ce-216843c6439b None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:05.422860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8847a75f-f4e0-4e4c-a3ce-216843c6439b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:05.425332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bd6c1-f709-4237-b79d-5f07627db146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:05.427732 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bd6c1-f709-4237-b79d-5f07627db146 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:05.428056 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bd6c1-f709-4237-b79d-5f07627db146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:05.428266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bd6c1-f709-4237-b79d-5f07627db146 admin admin] PATH_INFO: `/v3/projects/63f1324e656341079beb7b887bf9199f/users/287ee9f2edb24eda8053b3b9d15fff7c/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:05.491955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 903/1798] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:05.494024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04bd6c1-f709-4237-b79d-5f07627db146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63f1324e656341079beb7b887bf9199f, user_id=287ee9f2edb24eda8053b3b9d15fff7c, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:05.500271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847a75f-f4e0-4e4c-a3ce-216843c6439b None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:05.504605 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8847a75f-f4e0-4e4c-a3ce-216843c6439b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:05.527142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04bd6c1-f709-4237-b79d-5f07627db146 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:05.564260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847a75f-f4e0-4e4c-a3ce-216843c6439b None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:05.588974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:05.599771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 896/1799] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:05 2022] PUT /identity/v3/projects/63f1324e656341079beb7b887bf9199f/users/287ee9f2edb24eda8053b3b9d15fff7c/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:05.606681 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bd6c1-f709-4237-b79d-5f07627db146 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:05.767941 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8da2ca-28d3-4571-9547-41380d8048c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:05.769524 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8da2ca-28d3-4571-9547-41380d8048c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:05.774649 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bd6c1-f709-4237-b79d-5f07627db146 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:05.778253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8da2ca-28d3-4571-9547-41380d8048c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:05.779582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8da2ca-28d3-4571-9547-41380d8048c1 admin admin] PATH_INFO: `/v3/users/bef597abe146496698cce9c571228dc4` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:05.781696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c0540-1cbf-423d-a405-e2efec614ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:05.781846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8da2ca-28d3-4571-9547-41380d8048c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bef597abe146496698cce9c571228dc4)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:05.783666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c0540-1cbf-423d-a405-e2efec614ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:05.783954 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c0540-1cbf-423d-a405-e2efec614ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:05.784504 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c0540-1cbf-423d-a405-e2efec614ab5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:05.785033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c0540-1cbf-423d-a405-e2efec614ab5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:05.785345 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c0540-1cbf-423d-a405-e2efec614ab5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:05.786977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c0540-1cbf-423d-a405-e2efec614ab5 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:05.787983 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8da2ca-28d3-4571-9547-41380d8048c1 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:05.791197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:05.806127 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 897/1800] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:06:05.809213 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c0540-1cbf-423d-a405-e2efec614ab5 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:05.812973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-9c4c0540-1cbf-423d-a405-e2efec614ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:05.874864 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-bf8da2ca-28d3-4571-9547-41380d8048c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bef597abe146496698cce9c571228dc4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:05.893372 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-bf8da2ca-28d3-4571-9547-41380d8048c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bef597abe146496698cce9c571228dc4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:05.903885 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-bf8da2ca-28d3-4571-9547-41380d8048c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bef597abe146496698cce9c571228dc4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:05.915328 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-bf8da2ca-28d3-4571-9547-41380d8048c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bef597abe146496698cce9c571228dc4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:05.918395 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-bf8da2ca-28d3-4571-9547-41380d8048c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bef597abe146496698cce9c571228dc4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:05.927535 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 904/1801] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:05 2022] DELETE /identity/v3/users/bef597abe146496698cce9c571228dc4 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:05.928889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8da2ca-28d3-4571-9547-41380d8048c1 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:05.931333 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-bf8da2ca-28d3-4571-9547-41380d8048c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:06.079138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c0540-1cbf-423d-a405-e2efec614ab5 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:06.134572 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8da2ca-28d3-4571-9547-41380d8048c1 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:06.163083 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818fe39-116c-4bf0-bf77-6460baa8e346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:06.163918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818fe39-116c-4bf0-bf77-6460baa8e346 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:06.164302 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818fe39-116c-4bf0-bf77-6460baa8e346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:06.165261 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818fe39-116c-4bf0-bf77-6460baa8e346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:06.166117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818fe39-116c-4bf0-bf77-6460baa8e346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:06.172933 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818fe39-116c-4bf0-bf77-6460baa8e346 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:06.189871 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a7e6e-b9fe-45c5-9051-5b0366240537 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:06.191433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a7e6e-b9fe-45c5-9051-5b0366240537 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:06.192437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a7e6e-b9fe-45c5-9051-5b0366240537 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:06.192886 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a7e6e-b9fe-45c5-9051-5b0366240537 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:06.200978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1d0a7e6e-b9fe-45c5-9051-5b0366240537 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:06.209102 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 905/1802] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:06.213760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818fe39-116c-4bf0-bf77-6460baa8e346 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:06.219301 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5818fe39-116c-4bf0-bf77-6460baa8e346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:06.232083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a7e6e-b9fe-45c5-9051-5b0366240537 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:06.233516 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a7e6e-b9fe-45c5-9051-5b0366240537 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:06.252384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818fe39-116c-4bf0-bf77-6460baa8e346 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:06.263290 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 898/1803] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:06.265055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa9ce3-b59e-40ef-86ea-57fa79cee1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:06.279087 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a7e6e-b9fe-45c5-9051-5b0366240537 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:06.285383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa9ce3-b59e-40ef-86ea-57fa79cee1b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:06.287016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa9ce3-b59e-40ef-86ea-57fa79cee1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:06.287289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa9ce3-b59e-40ef-86ea-57fa79cee1b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:06.288139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa9ce3-b59e-40ef-86ea-57fa79cee1b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:06.294601 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa9ce3-b59e-40ef-86ea-57fa79cee1b0 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:06.315454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a7e6e-b9fe-45c5-9051-5b0366240537 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:06.338463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed8d11-e7ca-46d5-8bc9-eaa5f7a12291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:06.340985 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed8d11-e7ca-46d5-8bc9-eaa5f7a12291 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:06.341272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed8d11-e7ca-46d5-8bc9-eaa5f7a12291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:06.341565 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed8d11-e7ca-46d5-8bc9-eaa5f7a12291 admin admin] PATH_INFO: `/v3/projects/63f1324e656341079beb7b887bf9199f/users/287ee9f2edb24eda8053b3b9d15fff7c/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:06.355788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed8d11-e7ca-46d5-8bc9-eaa5f7a12291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63f1324e656341079beb7b887bf9199f, user_id=287ee9f2edb24eda8053b3b9d15fff7c, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:06.361385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed8d11-e7ca-46d5-8bc9-eaa5f7a12291 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:06.387667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 906/1804] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:06 2022] POST /identity/v3/projects => generated 371 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:06.393612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa9ce3-b59e-40ef-86ea-57fa79cee1b0 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:06.402880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 899/1805] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:06 2022] PUT /identity/v3/projects/63f1324e656341079beb7b887bf9199f/users/287ee9f2edb24eda8053b3b9d15fff7c/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:06.406669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:06.413809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bc218-6410-4582-9962-347b3a63283c None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:06.414461 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bc218-6410-4582-9962-347b3a63283c None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:06.414461 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bc218-6410-4582-9962-347b3a63283c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:06.471078 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-040bc218-6410-4582-9962-347b3a63283c None None] MFA Rules not processed for user `287ee9f2edb24eda8053b3b9d15fff7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:06.475208 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-040bc218-6410-4582-9962-347b3a63283c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:06.587421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa9ce3-b59e-40ef-86ea-57fa79cee1b0 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:06.658198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 900/1806] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:06.669337 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bc218-6410-4582-9962-347b3a63283c None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:06.670426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-040bc218-6410-4582-9962-347b3a63283c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:06.769544 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62536e2c-3fb6-4f1c-9397-cd92c7f6759a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:06.770311 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62536e2c-3fb6-4f1c-9397-cd92c7f6759a admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:06.770673 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62536e2c-3fb6-4f1c-9397-cd92c7f6759a admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:06.771033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62536e2c-3fb6-4f1c-9397-cd92c7f6759a admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:06.772014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62536e2c-3fb6-4f1c-9397-cd92c7f6759a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:06.772809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:06.777994 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62536e2c-3fb6-4f1c-9397-cd92c7f6759a admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:06.818716 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 907/1807] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:06 2022] POST /identity/v3/users => generated 286 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:06.820032 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62536e2c-3fb6-4f1c-9397-cd92c7f6759a admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:06.839949 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bc218-6410-4582-9962-347b3a63283c None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:06.846620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f12e5-3871-4dc8-a368-ac96d8f022f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:06.847066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f12e5-3871-4dc8-a368-ac96d8f022f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:06.847495 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f12e5-3871-4dc8-a368-ac96d8f022f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:06.847843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f12e5-3871-4dc8-a368-ac96d8f022f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:06.869128 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f12e5-3871-4dc8-a368-ac96d8f022f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:06.870808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f12e5-3871-4dc8-a368-ac96d8f022f7 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:06.871247 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62536e2c-3fb6-4f1c-9397-cd92c7f6759a admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:06.893877 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81641b2e-f946-4194-b9ac-24394e7f09ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:06.894571 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81641b2e-f946-4194-b9ac-24394e7f09ee admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:06.894855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81641b2e-f946-4194-b9ac-24394e7f09ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:06.895133 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81641b2e-f946-4194-b9ac-24394e7f09ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:06.895905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81641b2e-f946-4194-b9ac-24394e7f09ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:06.907829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81641b2e-f946-4194-b9ac-24394e7f09ee admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:06.926023 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 901/1808] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:06.973458 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 908/1809] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:06.978440 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f12e5-3871-4dc8-a368-ac96d8f022f7 service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:06.991140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f12e5-3871-4dc8-a368-ac96d8f022f7 service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:07.003246 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f5cdf-8576-41fe-97ae-50c7d8a984f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:07.004200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f5cdf-8576-41fe-97ae-50c7d8a984f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:07.004408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f5cdf-8576-41fe-97ae-50c7d8a984f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:07.004750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f5cdf-8576-41fe-97ae-50c7d8a984f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:07.005603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f5cdf-8576-41fe-97ae-50c7d8a984f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:07.007601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f5cdf-8576-41fe-97ae-50c7d8a984f5 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:07.041994 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 902/1810] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:07.043707 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81641b2e-f946-4194-b9ac-24394e7f09ee admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:07.072427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81641b2e-f946-4194-b9ac-24394e7f09ee admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:07.077920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f5cdf-8576-41fe-97ae-50c7d8a984f5 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:07.080069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1e9f5cdf-8576-41fe-97ae-50c7d8a984f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:07.095952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49add3d1-f22b-4985-b4b0-149c85533afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:07.099235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49add3d1-f22b-4985-b4b0-149c85533afe admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:07.099638 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49add3d1-f22b-4985-b4b0-149c85533afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:07.099928 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49add3d1-f22b-4985-b4b0-149c85533afe admin admin] PATH_INFO: `/v3/projects/9eea115b174e45a697351ae2e77b9dbb/users/355eec92657542739ecfc08d307379e5/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:07.112945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f5cdf-8576-41fe-97ae-50c7d8a984f5 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:07.162485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defe1b8-ceb6-4ece-8bdf-3ecf245d2fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:07.163639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defe1b8-ceb6-4ece-8bdf-3ecf245d2fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:07.163933 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defe1b8-ceb6-4ece-8bdf-3ecf245d2fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:07.164510 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defe1b8-ceb6-4ece-8bdf-3ecf245d2fb0 admin admin] PATH_INFO: `/v3/projects/e8db3288ce4743b49635435878722dfc` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:07.166105 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defe1b8-ceb6-4ece-8bdf-3ecf245d2fb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8db3288ce4743b49635435878722dfc)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:07.168086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defe1b8-ceb6-4ece-8bdf-3ecf245d2fb0 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:07.177895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49add3d1-f22b-4985-b4b0-149c85533afe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eea115b174e45a697351ae2e77b9dbb, user_id=355eec92657542739ecfc08d307379e5, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:07.182964 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49add3d1-f22b-4985-b4b0-149c85533afe admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:07.232146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 909/1811] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:07 2022] PUT /identity/v3/projects/9eea115b174e45a697351ae2e77b9dbb/users/355eec92657542739ecfc08d307379e5/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:07.234859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49add3d1-f22b-4985-b4b0-149c85533afe admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:07.237012 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-49add3d1-f22b-4985-b4b0-149c85533afe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:07.272048 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1defe1b8-ceb6-4ece-8bdf-3ecf245d2fb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8db3288ce4743b49635435878722dfc', 'request_id': 'req-1defe1b8-ceb6-4ece-8bdf-3ecf245d2fb0'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:07.285313 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1defe1b8-ceb6-4ece-8bdf-3ecf245d2fb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8db3288ce4743b49635435878722dfc', 'request_id': 'req-1defe1b8-ceb6-4ece-8bdf-3ecf245d2fb0'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:07.315770 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1defe1b8-ceb6-4ece-8bdf-3ecf245d2fb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8db3288ce4743b49635435878722dfc', 'request_id': 'req-1defe1b8-ceb6-4ece-8bdf-3ecf245d2fb0'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:07.346692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1defe1b8-ceb6-4ece-8bdf-3ecf245d2fb0 admin admin] The token cache is being invalidate because project e8db3288ce4743b49635435878722dfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:07.347141 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1defe1b8-ceb6-4ece-8bdf-3ecf245d2fb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:07.349745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1defe1b8-ceb6-4ece-8bdf-3ecf245d2fb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:07.350334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49add3d1-f22b-4985-b4b0-149c85533afe admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:07.357618 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 903/1812] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:07 2022] DELETE /identity/v3/projects/e8db3288ce4743b49635435878722dfc => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:07.359238 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e5fec-40ba-498a-a06c-045054ec5b93 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:07.359547 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e5fec-40ba-498a-a06c-045054ec5b93 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:07.359831 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e5fec-40ba-498a-a06c-045054ec5b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:07.371259 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:07.433762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-8d9e5fec-40ba-498a-a06c-045054ec5b93 None None] MFA Rules not processed for user `355eec92657542739ecfc08d307379e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:07.435625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8d9e5fec-40ba-498a-a06c-045054ec5b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:07.558024 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fce57d-55d6-4690-91ac-4d5945b80da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:07.562700 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fce57d-55d6-4690-91ac-4d5945b80da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:07.563017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fce57d-55d6-4690-91ac-4d5945b80da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:07.563356 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fce57d-55d6-4690-91ac-4d5945b80da9 admin admin] PATH_INFO: `/v3/projects/e2bd3999baa445ddbe68ccd26787bba5` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:07.573860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fce57d-55d6-4690-91ac-4d5945b80da9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2bd3999baa445ddbe68ccd26787bba5)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:07.575298 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 904/1813] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:07.577007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309cd47-9721-47c1-ae91-aa3476432a8f None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:07.577416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309cd47-9721-47c1-ae91-aa3476432a8f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:07.577787 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309cd47-9721-47c1-ae91-aa3476432a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:07.585945 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fce57d-55d6-4690-91ac-4d5945b80da9 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:07.606724 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-0309cd47-9721-47c1-ae91-aa3476432a8f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:07.606724 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0309cd47-9721-47c1-ae91-aa3476432a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:07.626813 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 905/1814] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:07.628354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309cd47-9721-47c1-ae91-aa3476432a8f None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:07.632008 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0309cd47-9721-47c1-ae91-aa3476432a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:07.653155 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309cd47-9721-47c1-ae91-aa3476432a8f None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:07.671424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99795d-09ec-4789-8cfe-836be2ee422b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:07.673327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99795d-09ec-4789-8cfe-836be2ee422b admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:07.677292 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99795d-09ec-4789-8cfe-836be2ee422b admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:07.677684 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99795d-09ec-4789-8cfe-836be2ee422b admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:07.678633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c99795d-09ec-4789-8cfe-836be2ee422b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:07.685036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c99795d-09ec-4789-8cfe-836be2ee422b admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:07.761798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 906/1815] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:07 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:07.761798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a8c71-7556-4bf4-9603-7b8a9e8b736b None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:07.761798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a8c71-7556-4bf4-9603-7b8a9e8b736b None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:07.761798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a8c71-7556-4bf4-9603-7b8a9e8b736b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:07.767151 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:07.786973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-fc1a8c71-7556-4bf4-9603-7b8a9e8b736b None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:07.790770 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-fc1a8c71-7556-4bf4-9603-7b8a9e8b736b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:07.804747 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c2fce57d-55d6-4690-91ac-4d5945b80da9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2bd3999baa445ddbe68ccd26787bba5', 'request_id': 'req-c2fce57d-55d6-4690-91ac-4d5945b80da9'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:07.822641 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c2fce57d-55d6-4690-91ac-4d5945b80da9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2bd3999baa445ddbe68ccd26787bba5', 'request_id': 'req-c2fce57d-55d6-4690-91ac-4d5945b80da9'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:07.844421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c2fce57d-55d6-4690-91ac-4d5945b80da9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2bd3999baa445ddbe68ccd26787bba5', 'request_id': 'req-c2fce57d-55d6-4690-91ac-4d5945b80da9'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:07.866353 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c2fce57d-55d6-4690-91ac-4d5945b80da9 admin admin] The token cache is being invalidate because project e2bd3999baa445ddbe68ccd26787bba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:07.866689 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c2fce57d-55d6-4690-91ac-4d5945b80da9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:07.868700 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c2fce57d-55d6-4690-91ac-4d5945b80da9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:07.870857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 910/1816] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:07 2022] DELETE /identity/v3/projects/e2bd3999baa445ddbe68ccd26787bba5 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:07.871714 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fce57d-55d6-4690-91ac-4d5945b80da9 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:07.873262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c2fce57d-55d6-4690-91ac-4d5945b80da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:08.002961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 907/1817] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:08.003990 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a8c71-7556-4bf4-9603-7b8a9e8b736b None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:08.009472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-fc1a8c71-7556-4bf4-9603-7b8a9e8b736b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:08.037047 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a8c71-7556-4bf4-9603-7b8a9e8b736b None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:08.047629 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07b07a-5f86-449c-8db8-9db3fc62fb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:08.048431 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07b07a-5f86-449c-8db8-9db3fc62fb51 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:08.048859 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07b07a-5f86-449c-8db8-9db3fc62fb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:08.049264 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07b07a-5f86-449c-8db8-9db3fc62fb51 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:08.050075 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd07b07a-5f86-449c-8db8-9db3fc62fb51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:08.053658 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd07b07a-5f86-449c-8db8-9db3fc62fb51 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:08.058819 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fce57d-55d6-4690-91ac-4d5945b80da9 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:08.071265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99327772-e480-40fe-b848-704bfe5ef707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:08.072628 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99327772-e480-40fe-b848-704bfe5ef707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:08.073076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99327772-e480-40fe-b848-704bfe5ef707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:08.073421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99327772-e480-40fe-b848-704bfe5ef707 admin admin] PATH_INFO: `/v3/users/31df89f3225f4f1db7a4699fe83cc2cf` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:08.074959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99327772-e480-40fe-b848-704bfe5ef707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31df89f3225f4f1db7a4699fe83cc2cf)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:08.077307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99327772-e480-40fe-b848-704bfe5ef707 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:08.082061 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 908/1818] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:08 2022] POST /identity/v3/users => generated 292 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:08.082061 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07b07a-5f86-449c-8db8-9db3fc62fb51 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:08.083372 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-fd07b07a-5f86-449c-8db8-9db3fc62fb51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:08.098001 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07b07a-5f86-449c-8db8-9db3fc62fb51 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:08.124153 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab8f74-2694-4135-932e-d94f0a9813c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:08.125525 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab8f74-2694-4135-932e-d94f0a9813c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:08.127120 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab8f74-2694-4135-932e-d94f0a9813c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:08.127414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab8f74-2694-4135-932e-d94f0a9813c2 admin admin] PATH_INFO: `/v3/users/f2de12ca8a2c470095facf741e0acabc` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:08.132100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab8f74-2694-4135-932e-d94f0a9813c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2de12ca8a2c470095facf741e0acabc)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:08.134575 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab8f74-2694-4135-932e-d94f0a9813c2 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:08.169825 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:08.173840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-99327772-e480-40fe-b848-704bfe5ef707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31df89f3225f4f1db7a4699fe83cc2cf'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:08.244948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-99327772-e480-40fe-b848-704bfe5ef707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31df89f3225f4f1db7a4699fe83cc2cf'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:08.263347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-99327772-e480-40fe-b848-704bfe5ef707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31df89f3225f4f1db7a4699fe83cc2cf'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:08.277285 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c7ab8f74-2694-4135-932e-d94f0a9813c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2de12ca8a2c470095facf741e0acabc'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:08.278013 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-99327772-e480-40fe-b848-704bfe5ef707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31df89f3225f4f1db7a4699fe83cc2cf'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:08.280178 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-99327772-e480-40fe-b848-704bfe5ef707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31df89f3225f4f1db7a4699fe83cc2cf'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:08.293243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c7ab8f74-2694-4135-932e-d94f0a9813c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2de12ca8a2c470095facf741e0acabc'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:08.304151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 911/1819] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:07 2022] DELETE /identity/v3/users/31df89f3225f4f1db7a4699fe83cc2cf => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:08.307190 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99327772-e480-40fe-b848-704bfe5ef707 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:08.307813 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c7ab8f74-2694-4135-932e-d94f0a9813c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2de12ca8a2c470095facf741e0acabc'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:08.308412 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c7ab8f74-2694-4135-932e-d94f0a9813c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2de12ca8a2c470095facf741e0acabc'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:08.309107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c7ab8f74-2694-4135-932e-d94f0a9813c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2de12ca8a2c470095facf741e0acabc'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:08.313587 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-99327772-e480-40fe-b848-704bfe5ef707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:08.344828 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 909/1820] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:08 2022] DELETE /identity/v3/users/f2de12ca8a2c470095facf741e0acabc => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:08.346683 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffc2ea-c2f8-4298-964c-dfd1909cb1db None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:08.346965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffc2ea-c2f8-4298-964c-dfd1909cb1db None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:08.347250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffc2ea-c2f8-4298-964c-dfd1909cb1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:08.418980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-a2ffc2ea-c2f8-4298-964c-dfd1909cb1db None None] MFA Rules not processed for user `8d6b823cc7dc434986f77739fa0259ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:08.420954 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a2ffc2ea-c2f8-4298-964c-dfd1909cb1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:08.477278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:08.514346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99327772-e480-40fe-b848-704bfe5ef707 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:08.589589 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 910/1821] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Jul 12 22:06:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:08.590544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffc2ea-c2f8-4298-964c-dfd1909cb1db None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:08.593049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a2ffc2ea-c2f8-4298-964c-dfd1909cb1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:08.701385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd034eea-49e5-4309-9deb-988b803a96db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:08.702094 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd034eea-49e5-4309-9deb-988b803a96db admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:08.702387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd034eea-49e5-4309-9deb-988b803a96db admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:08.702728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd034eea-49e5-4309-9deb-988b803a96db admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:08.703536 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd034eea-49e5-4309-9deb-988b803a96db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:08.704542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd034eea-49e5-4309-9deb-988b803a96db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:08.707180 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd034eea-49e5-4309-9deb-988b803a96db admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:08.734800 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 912/1822] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:06:08.734992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bb557-7c48-43c8-9ab0-a4000682472b None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:08.735186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bb557-7c48-43c8-9ab0-a4000682472b None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:08.735477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bb557-7c48-43c8-9ab0-a4000682472b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:08.783277 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-1a3bb557-7c48-43c8-9ab0-a4000682472b None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:08.783277 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-1a3bb557-7c48-43c8-9ab0-a4000682472b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:08.794518 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 913/1823] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:08.795436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bb557-7c48-43c8-9ab0-a4000682472b None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:08.797188 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-1a3bb557-7c48-43c8-9ab0-a4000682472b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:08.814201 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bb557-7c48-43c8-9ab0-a4000682472b None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:08.820120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ac7e9-a82a-4ffd-b58b-59c1f3041432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:08.821458 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ac7e9-a82a-4ffd-b58b-59c1f3041432 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:08.821832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ac7e9-a82a-4ffd-b58b-59c1f3041432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:08.823056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ac7e9-a82a-4ffd-b58b-59c1f3041432 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:08.824058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ac7e9-a82a-4ffd-b58b-59c1f3041432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:08.826650 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ac7e9-a82a-4ffd-b58b-59c1f3041432 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:08.830846 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffc2ea-c2f8-4298-964c-dfd1909cb1db None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:08.845980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0c595-afec-4912-8e74-f6951b61de9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:08.847018 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0c595-afec-4912-8e74-f6951b61de9e admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:08.847517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0c595-afec-4912-8e74-f6951b61de9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:08.847985 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0c595-afec-4912-8e74-f6951b61de9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:08.849135 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0c595-afec-4912-8e74-f6951b61de9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:08.859541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0c595-afec-4912-8e74-f6951b61de9e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:08.873549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 914/1824] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:08 2022] POST /identity/v3/projects => generated 407 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:08.874981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ac7e9-a82a-4ffd-b58b-59c1f3041432 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:08.884680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-cc7ac7e9-a82a-4ffd-b58b-59c1f3041432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:08.898120 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 911/1825] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:08.899554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0c595-afec-4912-8e74-f6951b61de9e admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:08.906482 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-5cf0c595-afec-4912-8e74-f6951b61de9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:08.999131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:09.047372 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ac7e9-a82a-4ffd-b58b-59c1f3041432 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:09.056184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fa6fe-a8f7-4b4b-b09c-8d4cc571f01e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:09.056921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fa6fe-a8f7-4b4b-b09c-8d4cc571f01e service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:09.057220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fa6fe-a8f7-4b4b-b09c-8d4cc571f01e service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:09.057551 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fa6fe-a8f7-4b4b-b09c-8d4cc571f01e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:09.124871 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0c595-afec-4912-8e74-f6951b61de9e admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:09.158379 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15fa6fe-a8f7-4b4b-b09c-8d4cc571f01e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:09.161321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15fa6fe-a8f7-4b4b-b09c-8d4cc571f01e service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:09.180074 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 915/1826] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:09.181540 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fa6fe-a8f7-4b4b-b09c-8d4cc571f01e service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:09.196294 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fa6fe-a8f7-4b4b-b09c-8d4cc571f01e service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:09.203113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:09.206771 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27163c54-a5f8-4155-a5f9-61ac3d5fea40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:09.208527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27163c54-a5f8-4155-a5f9-61ac3d5fea40 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:09.209163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27163c54-a5f8-4155-a5f9-61ac3d5fea40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:09.209854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27163c54-a5f8-4155-a5f9-61ac3d5fea40 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:09.210833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27163c54-a5f8-4155-a5f9-61ac3d5fea40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:09.226840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27163c54-a5f8-4155-a5f9-61ac3d5fea40 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:09.234016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c646f-e26f-46ed-a809-657d2fdabb39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837bb5ac18554dfdbbcc776689639494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:09.236470 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c646f-e26f-46ed-a809-657d2fdabb39 service swift] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:09.236836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c646f-e26f-46ed-a809-657d2fdabb39 service swift] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:09.237133 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c646f-e26f-46ed-a809-657d2fdabb39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:09.273496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 916/1827] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:09 2022] POST /identity/v3/users => generated 304 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:09.276299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27163c54-a5f8-4155-a5f9-61ac3d5fea40 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:09.302850 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27163c54-a5f8-4155-a5f9-61ac3d5fea40 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:09.320752 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ab40b-dbcc-41ae-a766-9f2a6a991e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:09.321990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ab40b-dbcc-41ae-a766-9f2a6a991e95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:09.322316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ab40b-dbcc-41ae-a766-9f2a6a991e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:09.322569 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ab40b-dbcc-41ae-a766-9f2a6a991e95 admin admin] PATH_INFO: `/v3/projects/9e813a59e84b4339a512d02b1096afa8/users/7859d4d1f41a4ac1b826295c786ebebb/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:09.356416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c646f-e26f-46ed-a809-657d2fdabb39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:09.358372 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c646f-e26f-46ed-a809-657d2fdabb39 service swift] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:09.359043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ab40b-dbcc-41ae-a766-9f2a6a991e95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e813a59e84b4339a512d02b1096afa8, user_id=7859d4d1f41a4ac1b826295c786ebebb, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:09.362566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ab40b-dbcc-41ae-a766-9f2a6a991e95 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:09.370550 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 912/1828] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Jul 12 22:06:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 22:06:09.371513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c646f-e26f-46ed-a809-657d2fdabb39 service swift] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:09.384393 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c646f-e26f-46ed-a809-657d2fdabb39 service swift] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:09.391333 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 917/1829] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:09 2022] PUT /identity/v3/projects/9e813a59e84b4339a512d02b1096afa8/users/7859d4d1f41a4ac1b826295c786ebebb/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:09.393937 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ab40b-dbcc-41ae-a766-9f2a6a991e95 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:09.498809 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ab40b-dbcc-41ae-a766-9f2a6a991e95 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:09.519848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342330b-8034-480b-b9ac-1008f67fbd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:09.524802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342330b-8034-480b-b9ac-1008f67fbd19 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:09.525123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342330b-8034-480b-b9ac-1008f67fbd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:09.526223 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342330b-8034-480b-b9ac-1008f67fbd19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:09.528642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342330b-8034-480b-b9ac-1008f67fbd19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:09.538504 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342330b-8034-480b-b9ac-1008f67fbd19 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:09.566789 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 918/1830] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:09.567932 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342330b-8034-480b-b9ac-1008f67fbd19 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:09.591278 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342330b-8034-480b-b9ac-1008f67fbd19 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:09.597565 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0ef81-26e6-40bc-998a-6536c7369bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:09.598789 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8489214-2478-4efd-99cb-fad4144e41c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:09.599453 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8489214-2478-4efd-99cb-fad4144e41c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:09.599713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8489214-2478-4efd-99cb-fad4144e41c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:09.600217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8489214-2478-4efd-99cb-fad4144e41c0 admin admin] PATH_INFO: `/v3/projects/9e813a59e84b4339a512d02b1096afa8/users/7859d4d1f41a4ac1b826295c786ebebb/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:09.602284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0ef81-26e6-40bc-998a-6536c7369bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:09.603002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0ef81-26e6-40bc-998a-6536c7369bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:09.604161 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0ef81-26e6-40bc-998a-6536c7369bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:09.605165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8489214-2478-4efd-99cb-fad4144e41c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e813a59e84b4339a512d02b1096afa8, user_id=7859d4d1f41a4ac1b826295c786ebebb, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:09.606320 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0ef81-26e6-40bc-998a-6536c7369bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:09.609940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0ef81-26e6-40bc-998a-6536c7369bd6 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:09.611885 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8489214-2478-4efd-99cb-fad4144e41c0 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:09.650693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 919/1831] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:09 2022] PUT /identity/v3/projects/9e813a59e84b4339a512d02b1096afa8/users/7859d4d1f41a4ac1b826295c786ebebb/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:09.653204 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777d64a-c8ab-435d-9580-21d89aa52df2 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:09.653770 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777d64a-c8ab-435d-9580-21d89aa52df2 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:09.657811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777d64a-c8ab-435d-9580-21d89aa52df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:09.658932 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 913/1832] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:09.660851 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0ef81-26e6-40bc-998a-6536c7369bd6 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:09.685985 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0ef81-26e6-40bc-998a-6536c7369bd6 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:09.742629 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-b777d64a-c8ab-435d-9580-21d89aa52df2 None None] MFA Rules not processed for user `7859d4d1f41a4ac1b826295c786ebebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:09.756943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b777d64a-c8ab-435d-9580-21d89aa52df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:09.795228 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:09.879936 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3896d-891e-44fb-a3e7-f36fa3ce7278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:09.880848 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3896d-891e-44fb-a3e7-f36fa3ce7278 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:09.881289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3896d-891e-44fb-a3e7-f36fa3ce7278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:09.881765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3896d-891e-44fb-a3e7-f36fa3ce7278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:09.884075 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3896d-891e-44fb-a3e7-f36fa3ce7278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:09.886844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3896d-891e-44fb-a3e7-f36fa3ce7278 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:09.910850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 914/1833] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:09.915792 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3896d-891e-44fb-a3e7-f36fa3ce7278 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:09.950986 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3896d-891e-44fb-a3e7-f36fa3ce7278 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:09.969710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185144e-13c4-4a1b-ab49-98e003a946e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:09.970832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185144e-13c4-4a1b-ab49-98e003a946e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:09.971238 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185144e-13c4-4a1b-ab49-98e003a946e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:09.971632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185144e-13c4-4a1b-ab49-98e003a946e9 admin admin] PATH_INFO: `/v3/projects/6c4e4a6f43d04d5d9cd3bea2ecea18c8/users/e5a3f9744cdf409ca2800cf2d7a799c2/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:09.996907 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 920/1834] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:09 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:10.008037 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777d64a-c8ab-435d-9580-21d89aa52df2 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:10.009691 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b777d64a-c8ab-435d-9580-21d89aa52df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:10.020539 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185144e-13c4-4a1b-ab49-98e003a946e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c4e4a6f43d04d5d9cd3bea2ecea18c8, user_id=e5a3f9744cdf409ca2800cf2d7a799c2, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:10.023054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185144e-13c4-4a1b-ab49-98e003a946e9 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:10.046387 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:10.059182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 915/1835] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:09 2022] PUT /identity/v3/projects/6c4e4a6f43d04d5d9cd3bea2ecea18c8/users/e5a3f9744cdf409ca2800cf2d7a799c2/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:10.078240 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd603db4-f621-4900-8346-bb60b0c1bdfd None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:10.078626 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd603db4-f621-4900-8346-bb60b0c1bdfd None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:10.078898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd603db4-f621-4900-8346-bb60b0c1bdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:10.185646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-bd603db4-f621-4900-8346-bb60b0c1bdfd None None] MFA Rules not processed for user `e5a3f9744cdf409ca2800cf2d7a799c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:10.186944 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-bd603db4-f621-4900-8346-bb60b0c1bdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:10.209475 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777d64a-c8ab-435d-9580-21d89aa52df2 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:10.449355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 916/1836] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:10 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:10.451701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd603db4-f621-4900-8346-bb60b0c1bdfd None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:10.453941 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e77bf-9736-4d29-85b5-1a2d07b71d01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:10.456485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e77bf-9736-4d29-85b5-1a2d07b71d01 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:10.457145 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e77bf-9736-4d29-85b5-1a2d07b71d01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:10.458183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e77bf-9736-4d29-85b5-1a2d07b71d01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:10.626308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd603db4-f621-4900-8346-bb60b0c1bdfd None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:10.643775 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d572d-1545-48b8-922e-a68191ac3b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:10.646530 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d572d-1545-48b8-922e-a68191ac3b61 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:10.646923 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d572d-1545-48b8-922e-a68191ac3b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:10.647315 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d572d-1545-48b8-922e-a68191ac3b61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:10.650098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d572d-1545-48b8-922e-a68191ac3b61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:10.655325 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d572d-1545-48b8-922e-a68191ac3b61 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:10.688440 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e77bf-9736-4d29-85b5-1a2d07b71d01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:10.690000 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e77bf-9736-4d29-85b5-1a2d07b71d01 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:10.700384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 917/1837] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:10 2022] POST /identity/v3/projects => generated 405 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:10.701565 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d572d-1545-48b8-922e-a68191ac3b61 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:10.704675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6b6d572d-1545-48b8-922e-a68191ac3b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:10.722153 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 921/1838] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:10 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 716 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:06:10.728927 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ec3fc-c74b-4ca0-ae0e-996500b1116a None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:10.728927 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ec3fc-c74b-4ca0-ae0e-996500b1116a None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:10.728927 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ec3fc-c74b-4ca0-ae0e-996500b1116a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:10.769543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-696ec3fc-c74b-4ca0-ae0e-996500b1116a None None] MFA Rules not processed for user `53b891a734a64b5983564e207de0359d`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:10.770860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-696ec3fc-c74b-4ca0-ae0e-996500b1116a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:10.822543 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d572d-1545-48b8-922e-a68191ac3b61 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:10.828155 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c589f5b-6af1-4a42-bd25-9cbd82613dfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:10.829143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c589f5b-6af1-4a42-bd25-9cbd82613dfa service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:10.829478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c589f5b-6af1-4a42-bd25-9cbd82613dfa service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:10.830141 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c589f5b-6af1-4a42-bd25-9cbd82613dfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:10.832905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-7c589f5b-6af1-4a42-bd25-9cbd82613dfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:10.834664 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:10.929489 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:10.975179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 922/1839] 10.222.0.27 () {58 vars in 1015 bytes} [Tue Jul 12 22:06:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:10.976054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ec3fc-c74b-4ca0-ae0e-996500b1116a None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:11.020281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c589f5b-6af1-4a42-bd25-9cbd82613dfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:11.022284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c589f5b-6af1-4a42-bd25-9cbd82613dfa service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:11.050783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ec3fc-c74b-4ca0-ae0e-996500b1116a None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:11.057748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403268e-5692-4f24-98cb-9293c5f3070c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:11.058702 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403268e-5692-4f24-98cb-9293c5f3070c admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:11.059162 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403268e-5692-4f24-98cb-9293c5f3070c admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:11.059328 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403268e-5692-4f24-98cb-9293c5f3070c admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:11.060068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403268e-5692-4f24-98cb-9293c5f3070c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:11.062170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403268e-5692-4f24-98cb-9293c5f3070c admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:11.086327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 918/1840] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:11.089380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c589f5b-6af1-4a42-bd25-9cbd82613dfa service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:11.093484 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-7c589f5b-6af1-4a42-bd25-9cbd82613dfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:11.103695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 923/1841] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:10 2022] POST /identity/v3/users => generated 309 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:11.104560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403268e-5692-4f24-98cb-9293c5f3070c admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:11.119148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c589f5b-6af1-4a42-bd25-9cbd82613dfa service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:11.129141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403268e-5692-4f24-98cb-9293c5f3070c admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:11.133353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed169461-3cee-4116-b375-7082873c063c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:11.134768 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed169461-3cee-4116-b375-7082873c063c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:11.135324 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed169461-3cee-4116-b375-7082873c063c admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:11.135828 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed169461-3cee-4116-b375-7082873c063c admin admin] PATH_INFO: `/v3/projects/5e72ec106cd04d4b9d7e51f144426f52` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:11.140268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b9b5f-19c7-460e-9084-ddafd42dcfc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:11.140972 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed169461-3cee-4116-b375-7082873c063c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e72ec106cd04d4b9d7e51f144426f52)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:11.141171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b9b5f-19c7-460e-9084-ddafd42dcfc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:11.141298 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b9b5f-19c7-460e-9084-ddafd42dcfc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:11.141578 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b9b5f-19c7-460e-9084-ddafd42dcfc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:11.142977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed169461-3cee-4116-b375-7082873c063c admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:11.146954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-fe8b9b5f-19c7-460e-9084-ddafd42dcfc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:11.297648 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ed169461-3cee-4116-b375-7082873c063c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e72ec106cd04d4b9d7e51f144426f52', 'request_id': 'req-ed169461-3cee-4116-b375-7082873c063c'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:11.314514 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ed169461-3cee-4116-b375-7082873c063c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e72ec106cd04d4b9d7e51f144426f52', 'request_id': 'req-ed169461-3cee-4116-b375-7082873c063c'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:11.325200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b9b5f-19c7-460e-9084-ddafd42dcfc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:11.326709 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b9b5f-19c7-460e-9084-ddafd42dcfc6 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:11.335459 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ed169461-3cee-4116-b375-7082873c063c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e72ec106cd04d4b9d7e51f144426f52', 'request_id': 'req-ed169461-3cee-4116-b375-7082873c063c'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:11.349553 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 924/1842] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:11.358100 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b9b5f-19c7-460e-9084-ddafd42dcfc6 service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:11.359908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ed169461-3cee-4116-b375-7082873c063c admin admin] The token cache is being invalidate because project 5e72ec106cd04d4b9d7e51f144426f52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:11.360590 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ed169461-3cee-4116-b375-7082873c063c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:11.364756 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ed169461-3cee-4116-b375-7082873c063c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:11.367524 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 919/1843] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:11 2022] DELETE /identity/v3/projects/5e72ec106cd04d4b9d7e51f144426f52 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:11.369971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed169461-3cee-4116-b375-7082873c063c admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:11.373808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ed169461-3cee-4116-b375-7082873c063c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:11.379187 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b9b5f-19c7-460e-9084-ddafd42dcfc6 service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:11.526522 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be889fc-fce6-49a4-a56b-e84bfd1ad24e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:11.527655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be889fc-fce6-49a4-a56b-e84bfd1ad24e service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:11.528091 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be889fc-fce6-49a4-a56b-e84bfd1ad24e service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:11.528583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be889fc-fce6-49a4-a56b-e84bfd1ad24e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:11.546929 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0be889fc-fce6-49a4-a56b-e84bfd1ad24e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:11.556654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed169461-3cee-4116-b375-7082873c063c admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:11.564616 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:11.572794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f343e95-9da3-4d66-aa0f-dc24b730ec16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:11.587629 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f343e95-9da3-4d66-aa0f-dc24b730ec16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:11.588427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f343e95-9da3-4d66-aa0f-dc24b730ec16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:11.588761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f343e95-9da3-4d66-aa0f-dc24b730ec16 admin admin] PATH_INFO: `/v3/projects/de74f612e99b4b3e8bc5d106db6a03c2` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:11.589030 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be889fc-fce6-49a4-a56b-e84bfd1ad24e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:11.593055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be889fc-fce6-49a4-a56b-e84bfd1ad24e service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:11.594475 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f343e95-9da3-4d66-aa0f-dc24b730ec16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de74f612e99b4b3e8bc5d106db6a03c2)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:11.599672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f343e95-9da3-4d66-aa0f-dc24b730ec16 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:11.621716 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 925/1844] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:11.625127 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be889fc-fce6-49a4-a56b-e84bfd1ad24e service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:11.630782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0be889fc-fce6-49a4-a56b-e84bfd1ad24e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:11.665354 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be889fc-fce6-49a4-a56b-e84bfd1ad24e service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:11.673828 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac757969-9fe9-46fd-b6fa-51f4104d238c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:11.674474 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac757969-9fe9-46fd-b6fa-51f4104d238c admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:11.674770 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac757969-9fe9-46fd-b6fa-51f4104d238c admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:11.675331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac757969-9fe9-46fd-b6fa-51f4104d238c admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:11.680602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac757969-9fe9-46fd-b6fa-51f4104d238c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:11.684579 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac757969-9fe9-46fd-b6fa-51f4104d238c admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:11.726751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 926/1845] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:11.731612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac757969-9fe9-46fd-b6fa-51f4104d238c admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:11.733079 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0f343e95-9da3-4d66-aa0f-dc24b730ec16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de74f612e99b4b3e8bc5d106db6a03c2', 'request_id': 'req-0f343e95-9da3-4d66-aa0f-dc24b730ec16'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:11.739528 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:11.740041 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ac757969-9fe9-46fd-b6fa-51f4104d238c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:11.769789 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac757969-9fe9-46fd-b6fa-51f4104d238c admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:11.793463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0f343e95-9da3-4d66-aa0f-dc24b730ec16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de74f612e99b4b3e8bc5d106db6a03c2', 'request_id': 'req-0f343e95-9da3-4d66-aa0f-dc24b730ec16'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:11.816515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0f343e95-9da3-4d66-aa0f-dc24b730ec16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de74f612e99b4b3e8bc5d106db6a03c2', 'request_id': 'req-0f343e95-9da3-4d66-aa0f-dc24b730ec16'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:11.855243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0f343e95-9da3-4d66-aa0f-dc24b730ec16 admin admin] The token cache is being invalidate because project de74f612e99b4b3e8bc5d106db6a03c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:11.855559 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0f343e95-9da3-4d66-aa0f-dc24b730ec16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:11.857608 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0f343e95-9da3-4d66-aa0f-dc24b730ec16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:11.863445 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 920/1846] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:11 2022] DELETE /identity/v3/projects/de74f612e99b4b3e8bc5d106db6a03c2 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:11.867872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f343e95-9da3-4d66-aa0f-dc24b730ec16 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:11.875532 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0f343e95-9da3-4d66-aa0f-dc24b730ec16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:11.973462 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f3c23-1f17-4f62-8e18-c0ff329efb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:11.980146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f3c23-1f17-4f62-8e18-c0ff329efb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:11.980529 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f3c23-1f17-4f62-8e18-c0ff329efb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:11.980849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f3c23-1f17-4f62-8e18-c0ff329efb51 admin admin] PATH_INFO: `/v3/users/287ee9f2edb24eda8053b3b9d15fff7c` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:11.985430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f3c23-1f17-4f62-8e18-c0ff329efb51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=287ee9f2edb24eda8053b3b9d15fff7c)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:11.990421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f3c23-1f17-4f62-8e18-c0ff329efb51 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:12.066786 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-495f3c23-1f17-4f62-8e18-c0ff329efb51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '287ee9f2edb24eda8053b3b9d15fff7c'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:12.076641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f343e95-9da3-4d66-aa0f-dc24b730ec16 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:12.086197 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-495f3c23-1f17-4f62-8e18-c0ff329efb51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '287ee9f2edb24eda8053b3b9d15fff7c'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:12.089957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded71f5-24fd-4116-9477-00f51f98098e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:12.092432 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded71f5-24fd-4116-9477-00f51f98098e admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:12.095246 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded71f5-24fd-4116-9477-00f51f98098e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:12.095752 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded71f5-24fd-4116-9477-00f51f98098e admin admin] PATH_INFO: `/v3/projects/e33ab7281bc34df9a35f2e7fe73d5b5e/users/442e2b7f02a24ff69f61ae94500c2e5f/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:12.097991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-495f3c23-1f17-4f62-8e18-c0ff329efb51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '287ee9f2edb24eda8053b3b9d15fff7c'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:12.110212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-495f3c23-1f17-4f62-8e18-c0ff329efb51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '287ee9f2edb24eda8053b3b9d15fff7c'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:12.113264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-495f3c23-1f17-4f62-8e18-c0ff329efb51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '287ee9f2edb24eda8053b3b9d15fff7c'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:12.129058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded71f5-24fd-4116-9477-00f51f98098e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e33ab7281bc34df9a35f2e7fe73d5b5e, user_id=442e2b7f02a24ff69f61ae94500c2e5f, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:12.136943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 927/1847] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:11 2022] DELETE /identity/v3/users/287ee9f2edb24eda8053b3b9d15fff7c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:12.138337 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded71f5-24fd-4116-9477-00f51f98098e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:12.139072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3a430-6ede-4a6a-85b0-7b9ff9fb6ff6 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:12.140176 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3a430-6ede-4a6a-85b0-7b9ff9fb6ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:12.140900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3a430-6ede-4a6a-85b0-7b9ff9fb6ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:12.169603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 921/1848] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:11 2022] PUT /identity/v3/projects/e33ab7281bc34df9a35f2e7fe73d5b5e/users/442e2b7f02a24ff69f61ae94500c2e5f/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:12.169942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-6dd3a430-6ede-4a6a-85b0-7b9ff9fb6ff6 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:12.171345 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-6dd3a430-6ede-4a6a-85b0-7b9ff9fb6ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:12.173843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded71f5-24fd-4116-9477-00f51f98098e admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:12.176479 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8ded71f5-24fd-4116-9477-00f51f98098e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:12.351839 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 928/1849] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:12.361293 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de612f-ec4d-40d1-8d65-6185b53a8321 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:12.361617 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de612f-ec4d-40d1-8d65-6185b53a8321 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:12.361858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de612f-ec4d-40d1-8d65-6185b53a8321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:12.393034 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:12.424004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-31de612f-ec4d-40d1-8d65-6185b53a8321 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:12.427271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-31de612f-ec4d-40d1-8d65-6185b53a8321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:12.455463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded71f5-24fd-4116-9477-00f51f98098e admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:12.456350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 929/1850] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:12.457765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de612f-ec4d-40d1-8d65-6185b53a8321 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:12.470075 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18771c8-15ac-404d-9d4e-3c7e132e7f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:12.470764 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18771c8-15ac-404d-9d4e-3c7e132e7f22 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:12.471092 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18771c8-15ac-404d-9d4e-3c7e132e7f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:12.471372 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18771c8-15ac-404d-9d4e-3c7e132e7f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:12.473072 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18771c8-15ac-404d-9d4e-3c7e132e7f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:12.477013 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18771c8-15ac-404d-9d4e-3c7e132e7f22 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:12.481541 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de612f-ec4d-40d1-8d65-6185b53a8321 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:12.494487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd346e-82ca-430a-89c0-d9767ac6b3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:12.502693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd346e-82ca-430a-89c0-d9767ac6b3ee admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:12.503975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd346e-82ca-430a-89c0-d9767ac6b3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:12.507041 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd346e-82ca-430a-89c0-d9767ac6b3ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:12.508055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd346e-82ca-430a-89c0-d9767ac6b3ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:12.511102 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd346e-82ca-430a-89c0-d9767ac6b3ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:12.517905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 922/1851] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:06:12.519555 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd346e-82ca-430a-89c0-d9767ac6b3ee admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:12.522038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18771c8-15ac-404d-9d4e-3c7e132e7f22 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:12.544866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18771c8-15ac-404d-9d4e-3c7e132e7f22 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:12.548354 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 930/1852] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:06:12.553294 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd346e-82ca-430a-89c0-d9767ac6b3ee admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:12.558548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d030f07-16c7-4889-b0bb-5b730ced4d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:12.559579 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d030f07-16c7-4889-b0bb-5b730ced4d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:12.559866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d030f07-16c7-4889-b0bb-5b730ced4d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:12.560147 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d030f07-16c7-4889-b0bb-5b730ced4d7c admin admin] PATH_INFO: `/v3/projects/e33ab7281bc34df9a35f2e7fe73d5b5e/users/442e2b7f02a24ff69f61ae94500c2e5f/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:12.566142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d030f07-16c7-4889-b0bb-5b730ced4d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e33ab7281bc34df9a35f2e7fe73d5b5e, user_id=442e2b7f02a24ff69f61ae94500c2e5f, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:12.571029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d030f07-16c7-4889-b0bb-5b730ced4d7c admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:12.576669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd346e-82ca-430a-89c0-d9767ac6b3ee admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:12.586354 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7a08a-8f4a-4a24-af89-5952007c03f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:12.587057 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7a08a-8f4a-4a24-af89-5952007c03f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:12.587351 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7a08a-8f4a-4a24-af89-5952007c03f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:12.587627 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7a08a-8f4a-4a24-af89-5952007c03f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:12.590365 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7a08a-8f4a-4a24-af89-5952007c03f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:12.593551 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7a08a-8f4a-4a24-af89-5952007c03f2 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:12.604141 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 923/1853] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:12 2022] PUT /identity/v3/projects/e33ab7281bc34df9a35f2e7fe73d5b5e/users/442e2b7f02a24ff69f61ae94500c2e5f/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:12.614401 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101b654-1af0-4d8e-ab7d-3e5014e372c3 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:12.614753 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101b654-1af0-4d8e-ab7d-3e5014e372c3 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:12.615327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101b654-1af0-4d8e-ab7d-3e5014e372c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:12.670364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 931/1854] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:12 2022] POST /identity/v3/projects => generated 357 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:12.674543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7a08a-8f4a-4a24-af89-5952007c03f2 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:12.677836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-3101b654-1af0-4d8e-ab7d-3e5014e372c3 None None] MFA Rules not processed for user `442e2b7f02a24ff69f61ae94500c2e5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:12.682963 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-3101b654-1af0-4d8e-ab7d-3e5014e372c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:12.767924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:12.878266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 924/1855] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:12 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:12.895392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7a08a-8f4a-4a24-af89-5952007c03f2 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:12.901307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f6e5b-c534-4705-84a0-ed5f9cea4eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:12.902146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f6e5b-c534-4705-84a0-ed5f9cea4eac admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:12.902432 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f6e5b-c534-4705-84a0-ed5f9cea4eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:12.902710 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f6e5b-c534-4705-84a0-ed5f9cea4eac admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:12.903612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f6e5b-c534-4705-84a0-ed5f9cea4eac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:12.905668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f6e5b-c534-4705-84a0-ed5f9cea4eac admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:12.938391 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101b654-1af0-4d8e-ab7d-3e5014e372c3 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:12.940296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-3101b654-1af0-4d8e-ab7d-3e5014e372c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:12.966751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 932/1856] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:12 2022] POST /identity/v3/users => generated 279 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:12.984581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f6e5b-c534-4705-84a0-ed5f9cea4eac admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:13.059387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f6e5b-c534-4705-84a0-ed5f9cea4eac admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:13.085264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc955e9-f1b9-4bc6-9fe2-3725f646fb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:13.102356 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc955e9-f1b9-4bc6-9fe2-3725f646fb79 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:13.104356 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc955e9-f1b9-4bc6-9fe2-3725f646fb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:13.105868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc955e9-f1b9-4bc6-9fe2-3725f646fb79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:13.114648 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101b654-1af0-4d8e-ab7d-3e5014e372c3 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:13.119868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba67df6-7253-4a1f-94df-694397a1d607 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:13.121662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba67df6-7253-4a1f-94df-694397a1d607 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:13.122282 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba67df6-7253-4a1f-94df-694397a1d607 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:13.122714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba67df6-7253-4a1f-94df-694397a1d607 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:13.131344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc955e9-f1b9-4bc6-9fe2-3725f646fb79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:13.144405 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc955e9-f1b9-4bc6-9fe2-3725f646fb79 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:13.162388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba67df6-7253-4a1f-94df-694397a1d607 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:13.169502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba67df6-7253-4a1f-94df-694397a1d607 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:13.222856 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 933/1857] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:13.224358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc955e9-f1b9-4bc6-9fe2-3725f646fb79 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:13.225725 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 925/1858] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:12 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:13.283052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc955e9-f1b9-4bc6-9fe2-3725f646fb79 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:13.305784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04a57b-54e3-4065-ac38-dade4f25cce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:13.306496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04a57b-54e3-4065-ac38-dade4f25cce7 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:13.306782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04a57b-54e3-4065-ac38-dade4f25cce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:13.307062 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04a57b-54e3-4065-ac38-dade4f25cce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:13.307812 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04a57b-54e3-4065-ac38-dade4f25cce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:13.317532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04a57b-54e3-4065-ac38-dade4f25cce7 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:13.364174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 934/1859] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:13.367140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba67df6-7253-4a1f-94df-694397a1d607 service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:13.397301 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba67df6-7253-4a1f-94df-694397a1d607 service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:13.423807 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc21dc-4c61-4a2d-8600-ec10850817a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:13.424877 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc21dc-4c61-4a2d-8600-ec10850817a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:13.425425 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc21dc-4c61-4a2d-8600-ec10850817a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:13.425715 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc21dc-4c61-4a2d-8600-ec10850817a6 admin admin] PATH_INFO: `/v3/projects/d9d67b22b9874e15933caef64ee8b4d3/users/b5dd9e216b7344bead337a573126e46d/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:13.505828 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc21dc-4c61-4a2d-8600-ec10850817a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9d67b22b9874e15933caef64ee8b4d3, user_id=b5dd9e216b7344bead337a573126e46d, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:13.509287 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc21dc-4c61-4a2d-8600-ec10850817a6 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:13.576816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 926/1860] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:13 2022] PUT /identity/v3/projects/d9d67b22b9874e15933caef64ee8b4d3/users/b5dd9e216b7344bead337a573126e46d/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:13.584269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6906f-59a3-420c-9b01-d50f604087c8 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:13.584840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6906f-59a3-420c-9b01-d50f604087c8 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:13.585479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6906f-59a3-420c-9b01-d50f604087c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:13.687878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-b7c6906f-59a3-420c-9b01-d50f604087c8 None None] MFA Rules not processed for user `b5dd9e216b7344bead337a573126e46d`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:13.697943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b7c6906f-59a3-420c-9b01-d50f604087c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:13.934200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc21dc-4c61-4a2d-8600-ec10850817a6 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:14.083499 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 935/1861] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:13 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:14.084612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6906f-59a3-420c-9b01-d50f604087c8 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:14.211008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:14.286543 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc21dc-4c61-4a2d-8600-ec10850817a6 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:14.310920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d74bc-6601-4a08-8b27-b788bf0a8f31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:14.311760 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d74bc-6601-4a08-8b27-b788bf0a8f31 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:14.312152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d74bc-6601-4a08-8b27-b788bf0a8f31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:14.312638 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d74bc-6601-4a08-8b27-b788bf0a8f31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:14.324749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-244d74bc-6601-4a08-8b27-b788bf0a8f31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:14.367615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6906f-59a3-420c-9b01-d50f604087c8 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:14.390130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba201ce-8302-4959-a8b8-0d51232f52ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:14.390828 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba201ce-8302-4959-a8b8-0d51232f52ea admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:14.391118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba201ce-8302-4959-a8b8-0d51232f52ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:14.391403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba201ce-8302-4959-a8b8-0d51232f52ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:14.392424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba201ce-8302-4959-a8b8-0d51232f52ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:14.394433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba201ce-8302-4959-a8b8-0d51232f52ea admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:14.488738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 936/1862] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:14 2022] POST /identity/v3/projects => generated 357 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:14.494765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba201ce-8302-4959-a8b8-0d51232f52ea admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:14.583386 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d74bc-6601-4a08-8b27-b788bf0a8f31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:14.588145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d74bc-6601-4a08-8b27-b788bf0a8f31 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:14.642695 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 927/1863] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:13 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 709 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:06:14.651922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f6c41-5f6b-43a8-9586-19a07257c292 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:14.652775 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f6c41-5f6b-43a8-9586-19a07257c292 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:14.653362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f6c41-5f6b-43a8-9586-19a07257c292 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:14.655800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 928/1864] 10.222.0.44 () {54 vars in 921 bytes} [Tue Jul 12 22:06:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 22:06:14.672999 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b029d02-7aaf-428d-ac5e-eb15b771aa22 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:14.673406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b029d02-7aaf-428d-ac5e-eb15b771aa22 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:14.673794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b029d02-7aaf-428d-ac5e-eb15b771aa22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:14.705238 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:14.738915 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba201ce-8302-4959-a8b8-0d51232f52ea admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:14.747223 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-9b029d02-7aaf-428d-ac5e-eb15b771aa22 None None] MFA Rules not processed for user `24915625354f418eb7bc4d5ada85fdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:14.748619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-9b029d02-7aaf-428d-ac5e-eb15b771aa22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:14.753042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5e093-0e8e-46cb-8081-8ae5a0c4a4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:14.753786 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5e093-0e8e-46cb-8081-8ae5a0c4a4bf admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:14.754078 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5e093-0e8e-46cb-8081-8ae5a0c4a4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:14.754392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5e093-0e8e-46cb-8081-8ae5a0c4a4bf admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:14.755395 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec5e093-0e8e-46cb-8081-8ae5a0c4a4bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:14.757239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec5e093-0e8e-46cb-8081-8ae5a0c4a4bf admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:14.819965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 937/1865] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:14 2022] POST /identity/v3/users => generated 285 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:14.828577 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5e093-0e8e-46cb-8081-8ae5a0c4a4bf admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:14.860217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5e093-0e8e-46cb-8081-8ae5a0c4a4bf admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:14.882519 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77238b6c-38bd-4ed7-abdf-e863ff7cd606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:14.884200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77238b6c-38bd-4ed7-abdf-e863ff7cd606 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:14.884573 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77238b6c-38bd-4ed7-abdf-e863ff7cd606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:14.884973 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77238b6c-38bd-4ed7-abdf-e863ff7cd606 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:14.885840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77238b6c-38bd-4ed7-abdf-e863ff7cd606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:14.887528 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77238b6c-38bd-4ed7-abdf-e863ff7cd606 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:14.933048 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 938/1866] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:06:14.935861 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77238b6c-38bd-4ed7-abdf-e863ff7cd606 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:14.947602 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 929/1867] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Jul 12 22:06:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:14.985819 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77238b6c-38bd-4ed7-abdf-e863ff7cd606 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:15.002080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa09079-3c12-4a6e-84a1-11bcc0a1fce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:15.003338 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa09079-3c12-4a6e-84a1-11bcc0a1fce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:15.004625 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa09079-3c12-4a6e-84a1-11bcc0a1fce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:15.005124 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa09079-3c12-4a6e-84a1-11bcc0a1fce8 admin admin] PATH_INFO: `/v3/projects/fd58b84b67ae46dd9790ee9314e404b1/users/1a3c393b00e34c3b92d4e46570b1b96a/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:15.052386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa09079-3c12-4a6e-84a1-11bcc0a1fce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd58b84b67ae46dd9790ee9314e404b1, user_id=1a3c393b00e34c3b92d4e46570b1b96a, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:15.056439 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa09079-3c12-4a6e-84a1-11bcc0a1fce8 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:15.089400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 939/1868] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:14 2022] PUT /identity/v3/projects/fd58b84b67ae46dd9790ee9314e404b1/users/1a3c393b00e34c3b92d4e46570b1b96a/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:15.093845 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b029d02-7aaf-428d-ac5e-eb15b771aa22 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:15.128205 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b029d02-7aaf-428d-ac5e-eb15b771aa22 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:15.291920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ebdac-a2fd-468a-81c3-3765e868c9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:15.292883 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ebdac-a2fd-468a-81c3-3765e868c9e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:15.293179 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ebdac-a2fd-468a-81c3-3765e868c9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:15.293544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ebdac-a2fd-468a-81c3-3765e868c9e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:15.294547 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ebdac-a2fd-468a-81c3-3765e868c9e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:15.307573 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ebdac-a2fd-468a-81c3-3765e868c9e1 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:15.335395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 930/1869] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:15.339810 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa09079-3c12-4a6e-84a1-11bcc0a1fce8 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:15.356351 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa09079-3c12-4a6e-84a1-11bcc0a1fce8 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:15.369274 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c35a91-1c74-47e8-a953-2fdffa32c688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:15.372671 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c35a91-1c74-47e8-a953-2fdffa32c688 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:15.373231 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c35a91-1c74-47e8-a953-2fdffa32c688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:15.375016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c35a91-1c74-47e8-a953-2fdffa32c688 admin admin] PATH_INFO: `/v3/projects/fd58b84b67ae46dd9790ee9314e404b1/users/1a3c393b00e34c3b92d4e46570b1b96a/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:15.379309 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c35a91-1c74-47e8-a953-2fdffa32c688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd58b84b67ae46dd9790ee9314e404b1, user_id=1a3c393b00e34c3b92d4e46570b1b96a, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:15.387615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c35a91-1c74-47e8-a953-2fdffa32c688 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:15.432221 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 940/1870] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:15 2022] PUT /identity/v3/projects/fd58b84b67ae46dd9790ee9314e404b1/users/1a3c393b00e34c3b92d4e46570b1b96a/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:15.445351 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501d460-2679-4c74-ad70-ead694f1b876 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:15.447367 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501d460-2679-4c74-ad70-ead694f1b876 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:15.454642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501d460-2679-4c74-ad70-ead694f1b876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:15.534053 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-2501d460-2679-4c74-ad70-ead694f1b876 None None] MFA Rules not processed for user `1a3c393b00e34c3b92d4e46570b1b96a`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:15.535522 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-2501d460-2679-4c74-ad70-ead694f1b876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:15.675256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c35a91-1c74-47e8-a953-2fdffa32c688 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:15.678685 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e2c35a91-1c74-47e8-a953-2fdffa32c688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:15.807652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:15.869981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 931/1871] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:15 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:15.897387 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501d460-2679-4c74-ad70-ead694f1b876 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:15.899283 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-2501d460-2679-4c74-ad70-ead694f1b876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:15.913615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c35a91-1c74-47e8-a953-2fdffa32c688 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:15.947748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8796649-1f38-4f61-b5e5-013a8515370e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:15.948688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8796649-1f38-4f61-b5e5-013a8515370e service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:15.949167 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8796649-1f38-4f61-b5e5-013a8515370e service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:15.949611 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8796649-1f38-4f61-b5e5-013a8515370e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:15.959744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:16.045573 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8796649-1f38-4f61-b5e5-013a8515370e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:16.057334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8796649-1f38-4f61-b5e5-013a8515370e service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:16.099811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 941/1872] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:06:16.102435 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8796649-1f38-4f61-b5e5-013a8515370e service nova] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:16.109011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c8796649-1f38-4f61-b5e5-013a8515370e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:16.136866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501d460-2679-4c74-ad70-ead694f1b876 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:16.165681 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd95515-13c6-4456-a7c3-62dac4da378a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:16.166852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd95515-13c6-4456-a7c3-62dac4da378a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:16.167414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd95515-13c6-4456-a7c3-62dac4da378a admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:16.170047 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd95515-13c6-4456-a7c3-62dac4da378a admin admin] PATH_INFO: `/v3/users/73ae5f51260447c2b2799d6ff88841fc` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:16.173540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd95515-13c6-4456-a7c3-62dac4da378a admin admin] RBAC: Authorizing `identity:delete_user(user_id=73ae5f51260447c2b2799d6ff88841fc)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:16.183470 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd95515-13c6-4456-a7c3-62dac4da378a admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:16.290837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8796649-1f38-4f61-b5e5-013a8515370e service nova] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:16.296924 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c5aca-8e28-405e-89e3-ca66bb93662d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:16.298352 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c5aca-8e28-405e-89e3-ca66bb93662d service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:16.298877 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c5aca-8e28-405e-89e3-ca66bb93662d service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:16.299425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c5aca-8e28-405e-89e3-ca66bb93662d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:16.303677 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0fd95515-13c6-4456-a7c3-62dac4da378a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73ae5f51260447c2b2799d6ff88841fc'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:16.315040 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c5aca-8e28-405e-89e3-ca66bb93662d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:16.315674 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0fd95515-13c6-4456-a7c3-62dac4da378a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73ae5f51260447c2b2799d6ff88841fc'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:16.317677 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c5aca-8e28-405e-89e3-ca66bb93662d service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:16.332286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0fd95515-13c6-4456-a7c3-62dac4da378a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73ae5f51260447c2b2799d6ff88841fc'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:16.333601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0fd95515-13c6-4456-a7c3-62dac4da378a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73ae5f51260447c2b2799d6ff88841fc'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:16.335599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 942/1873] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:16 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:16.338138 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd84e4d-a9d7-41e6-b988-0276974f985e None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:16.340472 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd84e4d-a9d7-41e6-b988-0276974f985e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:16.341543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd84e4d-a9d7-41e6-b988-0276974f985e None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:16.345698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0fd95515-13c6-4456-a7c3-62dac4da378a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73ae5f51260447c2b2799d6ff88841fc'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:16.346696 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 943/1874] 10.222.0.44 () {54 vars in 921 bytes} [Tue Jul 12 22:06:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:06:16.407709 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 932/1875] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:15 2022] DELETE /identity/v3/users/73ae5f51260447c2b2799d6ff88841fc => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:16.413181 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc41500-f7bb-459f-85cc-1f0b0ac54128 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:16.413723 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc41500-f7bb-459f-85cc-1f0b0ac54128 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:16.414746 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc41500-f7bb-459f-85cc-1f0b0ac54128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:16.452921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-9fc41500-f7bb-459f-85cc-1f0b0ac54128 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:16.454716 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-9fc41500-f7bb-459f-85cc-1f0b0ac54128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:16.473667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd95515-13c6-4456-a7c3-62dac4da378a admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:16.475647 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0fd95515-13c6-4456-a7c3-62dac4da378a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:16.597868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 944/1876] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:16.612043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc41500-f7bb-459f-85cc-1f0b0ac54128 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:16.618123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-9fc41500-f7bb-459f-85cc-1f0b0ac54128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:16.657855 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd95515-13c6-4456-a7c3-62dac4da378a admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:16.677881 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca381f-43c3-4176-9def-7f0a6f555bc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:16.678697 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca381f-43c3-4176-9def-7f0a6f555bc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:16.679081 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca381f-43c3-4176-9def-7f0a6f555bc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:16.679310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca381f-43c3-4176-9def-7f0a6f555bc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:16.683272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-77ca381f-43c3-4176-9def-7f0a6f555bc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:16.742757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc41500-f7bb-459f-85cc-1f0b0ac54128 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:16.749313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69008c-e85d-4ddc-bd2e-de61a079a8ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:16.750935 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69008c-e85d-4ddc-bd2e-de61a079a8ae service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:16.751385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69008c-e85d-4ddc-bd2e-de61a079a8ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:16.751674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69008c-e85d-4ddc-bd2e-de61a079a8ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:16.768739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69008c-e85d-4ddc-bd2e-de61a079a8ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:16.770777 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69008c-e85d-4ddc-bd2e-de61a079a8ae service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:16.775262 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:16.792554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca381f-43c3-4176-9def-7f0a6f555bc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:16.794897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca381f-43c3-4176-9def-7f0a6f555bc6 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:16.801763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 945/1877] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:16.804199 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65be87-29cb-4d38-977f-c2876db7cb04 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:16.804802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65be87-29cb-4d38-977f-c2876db7cb04 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:16.805364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65be87-29cb-4d38-977f-c2876db7cb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:16.837209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 933/1878] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:16 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:06:16.838552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca381f-43c3-4176-9def-7f0a6f555bc6 service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:16.839915 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-dd65be87-29cb-4d38-977f-c2876db7cb04 None None] MFA Rules not processed for user `24915625354f418eb7bc4d5ada85fdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:16.841327 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-dd65be87-29cb-4d38-977f-c2876db7cb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:16.844976 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-77ca381f-43c3-4176-9def-7f0a6f555bc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:16.867110 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:16.887435 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca381f-43c3-4176-9def-7f0a6f555bc6 service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:16.903698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de0226-fd43-4825-ad4c-8f78587d2d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:16.905145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de0226-fd43-4825-ad4c-8f78587d2d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:16.905440 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de0226-fd43-4825-ad4c-8f78587d2d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:16.907301 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de0226-fd43-4825-ad4c-8f78587d2d81 admin admin] PATH_INFO: `/v3/projects/63f1324e656341079beb7b887bf9199f` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:16.912973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de0226-fd43-4825-ad4c-8f78587d2d81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63f1324e656341079beb7b887bf9199f)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:16.918744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de0226-fd43-4825-ad4c-8f78587d2d81 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:17.048889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 946/1879] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Jul 12 22:06:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:17.081354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a6de0226-fd43-4825-ad4c-8f78587d2d81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63f1324e656341079beb7b887bf9199f', 'request_id': 'req-a6de0226-fd43-4825-ad4c-8f78587d2d81'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:17.089858 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a6de0226-fd43-4825-ad4c-8f78587d2d81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63f1324e656341079beb7b887bf9199f', 'request_id': 'req-a6de0226-fd43-4825-ad4c-8f78587d2d81'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:17.107366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a6de0226-fd43-4825-ad4c-8f78587d2d81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63f1324e656341079beb7b887bf9199f', 'request_id': 'req-a6de0226-fd43-4825-ad4c-8f78587d2d81'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:17.123253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a6de0226-fd43-4825-ad4c-8f78587d2d81 admin admin] The token cache is being invalidate because project 63f1324e656341079beb7b887bf9199f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:17.123814 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a6de0226-fd43-4825-ad4c-8f78587d2d81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:17.124773 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a6de0226-fd43-4825-ad4c-8f78587d2d81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:17.130214 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 934/1880] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:16 2022] DELETE /identity/v3/projects/63f1324e656341079beb7b887bf9199f => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:17.133114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f87fc-c3d3-441d-99d3-7abb3d882462 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:17.133114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f87fc-c3d3-441d-99d3-7abb3d882462 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:17.133718 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f87fc-c3d3-441d-99d3-7abb3d882462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:17.159626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-e18f87fc-c3d3-441d-99d3-7abb3d882462 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:17.169412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e18f87fc-c3d3-441d-99d3-7abb3d882462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:17.194767 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de0226-fd43-4825-ad4c-8f78587d2d81 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:17.198865 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a6de0226-fd43-4825-ad4c-8f78587d2d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:17.284369 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 947/1881] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:17.293986 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3016c-12bd-4b9a-9266-a083d1c5c2ff None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:17.294739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3016c-12bd-4b9a-9266-a083d1c5c2ff None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:17.295191 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3016c-12bd-4b9a-9266-a083d1c5c2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:17.354211 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-58a3016c-12bd-4b9a-9266-a083d1c5c2ff None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:17.356301 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-58a3016c-12bd-4b9a-9266-a083d1c5c2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:17.367314 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 948/1882] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:17.368930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de0226-fd43-4825-ad4c-8f78587d2d81 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:17.373594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3016c-12bd-4b9a-9266-a083d1c5c2ff None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:17.375564 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecac6d-cc2b-47b8-9bf6-d75dc8d41d81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:17.376235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecac6d-cc2b-47b8-9bf6-d75dc8d41d81 service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:17.376518 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecac6d-cc2b-47b8-9bf6-d75dc8d41d81 service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:17.380149 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ecac6d-cc2b-47b8-9bf6-d75dc8d41d81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:17.380149 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-72ecac6d-cc2b-47b8-9bf6-d75dc8d41d81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:17.392576 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecac6d-cc2b-47b8-9bf6-d75dc8d41d81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:17.394443 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3016c-12bd-4b9a-9266-a083d1c5c2ff None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:17.394584 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ecac6d-cc2b-47b8-9bf6-d75dc8d41d81 service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:17.404680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6c7ba-8b8f-49a2-bfb0-97bb2129ee11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:17.405747 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6c7ba-8b8f-49a2-bfb0-97bb2129ee11 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:17.406973 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6c7ba-8b8f-49a2-bfb0-97bb2129ee11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:17.407419 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6c7ba-8b8f-49a2-bfb0-97bb2129ee11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:17.408391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6c7ba-8b8f-49a2-bfb0-97bb2129ee11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:17.408903 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6c7ba-8b8f-49a2-bfb0-97bb2129ee11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:17.416022 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6c7ba-8b8f-49a2-bfb0-97bb2129ee11 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:17.418970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 935/1883] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:06:17.448691 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 949/1884] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:06:17.450005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecac6d-cc2b-47b8-9bf6-d75dc8d41d81 service nova] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:17.462731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ecac6d-cc2b-47b8-9bf6-d75dc8d41d81 service nova] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:17.468635 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1d2fb-1111-471d-bf1e-0ae6d9b38fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:17.469549 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1d2fb-1111-471d-bf1e-0ae6d9b38fdf admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:17.470094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1d2fb-1111-471d-bf1e-0ae6d9b38fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:17.470554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1d2fb-1111-471d-bf1e-0ae6d9b38fdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:17.471476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c1d2fb-1111-471d-bf1e-0ae6d9b38fdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:17.473878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c1d2fb-1111-471d-bf1e-0ae6d9b38fdf admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:17.512197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 936/1885] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:17 2022] POST /identity/v3/projects => generated 367 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:06:17.516444 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6c7ba-8b8f-49a2-bfb0-97bb2129ee11 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:17.534973 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6c7ba-8b8f-49a2-bfb0-97bb2129ee11 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:17.547965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1d2fb-1111-471d-bf1e-0ae6d9b38fdf admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:17.549597 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-17c1d2fb-1111-471d-bf1e-0ae6d9b38fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:17.620228 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:17.626827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c72f0c-eb4d-4321-a6da-d5b80add1e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:17.627589 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c72f0c-eb4d-4321-a6da-d5b80add1e6e admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:17.627919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c72f0c-eb4d-4321-a6da-d5b80add1e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:17.628288 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c72f0c-eb4d-4321-a6da-d5b80add1e6e admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:17.629376 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c72f0c-eb4d-4321-a6da-d5b80add1e6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:17.632593 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c72f0c-eb4d-4321-a6da-d5b80add1e6e admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:17.647627 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1d2fb-1111-471d-bf1e-0ae6d9b38fdf admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:17.654696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842ab15-c926-4abb-b434-6da6cb8389b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:17.655588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842ab15-c926-4abb-b434-6da6cb8389b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:17.655876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842ab15-c926-4abb-b434-6da6cb8389b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:17.656157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842ab15-c926-4abb-b434-6da6cb8389b5 admin admin] PATH_INFO: `/v3/projects/f46254a3fea94bf19cacba55988e4c49` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:17.657523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c842ab15-c926-4abb-b434-6da6cb8389b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f46254a3fea94bf19cacba55988e4c49)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:17.661351 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 950/1886] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:17 2022] POST /identity/v3/users => generated 284 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:17.661522 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c842ab15-c926-4abb-b434-6da6cb8389b5 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:17.663089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c72f0c-eb4d-4321-a6da-d5b80add1e6e admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:17.680568 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c72f0c-eb4d-4321-a6da-d5b80add1e6e admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:17.685772 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d73829-45e6-44e1-a5bf-f371ec39e47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:17.686450 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d73829-45e6-44e1-a5bf-f371ec39e47c admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:17.687039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d73829-45e6-44e1-a5bf-f371ec39e47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:17.687316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d73829-45e6-44e1-a5bf-f371ec39e47c admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:17.688053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d73829-45e6-44e1-a5bf-f371ec39e47c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:17.689951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d73829-45e6-44e1-a5bf-f371ec39e47c admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:17.705279 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 951/1887] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:17.706301 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d73829-45e6-44e1-a5bf-f371ec39e47c admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:17.718665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d73829-45e6-44e1-a5bf-f371ec39e47c admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:17.719429 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c842ab15-c926-4abb-b434-6da6cb8389b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f46254a3fea94bf19cacba55988e4c49', 'request_id': 'req-c842ab15-c926-4abb-b434-6da6cb8389b5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:17.724688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:17.727938 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c842ab15-c926-4abb-b434-6da6cb8389b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f46254a3fea94bf19cacba55988e4c49', 'request_id': 'req-c842ab15-c926-4abb-b434-6da6cb8389b5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:17.738456 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c842ab15-c926-4abb-b434-6da6cb8389b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f46254a3fea94bf19cacba55988e4c49', 'request_id': 'req-c842ab15-c926-4abb-b434-6da6cb8389b5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:17.754858 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c842ab15-c926-4abb-b434-6da6cb8389b5 admin admin] The token cache is being invalidate because project f46254a3fea94bf19cacba55988e4c49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:17.755548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c842ab15-c926-4abb-b434-6da6cb8389b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:17.755843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c842ab15-c926-4abb-b434-6da6cb8389b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:17.757470 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 937/1888] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:17 2022] DELETE /identity/v3/projects/f46254a3fea94bf19cacba55988e4c49 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:17.761557 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842ab15-c926-4abb-b434-6da6cb8389b5 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:17.763066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c842ab15-c926-4abb-b434-6da6cb8389b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:17.785209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4879369-9564-49ee-81c9-bf556084c0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:17.786105 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4879369-9564-49ee-81c9-bf556084c0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:17.786218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4879369-9564-49ee-81c9-bf556084c0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:17.786430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4879369-9564-49ee-81c9-bf556084c0a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:17.787398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4879369-9564-49ee-81c9-bf556084c0a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:17.789689 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4879369-9564-49ee-81c9-bf556084c0a5 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:17.800289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 952/1889] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:06:17.804412 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4879369-9564-49ee-81c9-bf556084c0a5 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:17.806817 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-f4879369-9564-49ee-81c9-bf556084c0a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:17.819934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4879369-9564-49ee-81c9-bf556084c0a5 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:17.825369 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9bd50-2027-45e4-b118-5d9795c796d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:17.826259 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9bd50-2027-45e4-b118-5d9795c796d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:17.826539 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9bd50-2027-45e4-b118-5d9795c796d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:17.826813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9bd50-2027-45e4-b118-5d9795c796d5 admin admin] PATH_INFO: `/v3/projects/d9e5b9f0a1d84dc78ead517eb1533265/users/b58340f1f571471fbb535968cc4f1786/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:17.842895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9bd50-2027-45e4-b118-5d9795c796d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e5b9f0a1d84dc78ead517eb1533265, user_id=b58340f1f571471fbb535968cc4f1786, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:17.843123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842ab15-c926-4abb-b434-6da6cb8389b5 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:17.845589 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9bd50-2027-45e4-b118-5d9795c796d5 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:17.847717 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4d314-125e-4d98-8f1e-a68fb4ae67e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:17.848599 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4d314-125e-4d98-8f1e-a68fb4ae67e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:17.848914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4d314-125e-4d98-8f1e-a68fb4ae67e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:17.849233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4d314-125e-4d98-8f1e-a68fb4ae67e0 admin admin] PATH_INFO: `/v3/users/0c868c41878248ba9a934b50fdc1fa82` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:17.850584 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4d314-125e-4d98-8f1e-a68fb4ae67e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c868c41878248ba9a934b50fdc1fa82)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:17.853453 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4d314-125e-4d98-8f1e-a68fb4ae67e0 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:17.860151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 953/1890] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:17 2022] PUT /identity/v3/projects/d9e5b9f0a1d84dc78ead517eb1533265/users/b58340f1f571471fbb535968cc4f1786/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:17.870654 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0332f-bc6a-49ae-950e-7969d9332806 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:17.871023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0332f-bc6a-49ae-950e-7969d9332806 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:17.871403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0332f-bc6a-49ae-950e-7969d9332806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:17.903953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-82c0332f-bc6a-49ae-950e-7969d9332806 None None] MFA Rules not processed for user `b58340f1f571471fbb535968cc4f1786`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:17.906513 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-82c0332f-bc6a-49ae-950e-7969d9332806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:17.919507 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-cbf4d314-125e-4d98-8f1e-a68fb4ae67e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c868c41878248ba9a934b50fdc1fa82'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:17.927000 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-cbf4d314-125e-4d98-8f1e-a68fb4ae67e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c868c41878248ba9a934b50fdc1fa82'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:17.937040 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-cbf4d314-125e-4d98-8f1e-a68fb4ae67e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c868c41878248ba9a934b50fdc1fa82'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:17.940810 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-cbf4d314-125e-4d98-8f1e-a68fb4ae67e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c868c41878248ba9a934b50fdc1fa82'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:17.940810 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-cbf4d314-125e-4d98-8f1e-a68fb4ae67e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c868c41878248ba9a934b50fdc1fa82'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:17.949158 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:17.977366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 938/1891] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:17 2022] DELETE /identity/v3/users/0c868c41878248ba9a934b50fdc1fa82 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:18.004517 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 954/1892] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:18.010580 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4d314-125e-4d98-8f1e-a68fb4ae67e0 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:18.013281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-cbf4d314-125e-4d98-8f1e-a68fb4ae67e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:18.103492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4d314-125e-4d98-8f1e-a68fb4ae67e0 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:18.109260 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887dab2d-9876-44b6-a650-4d758e4f373c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:18.110037 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887dab2d-9876-44b6-a650-4d758e4f373c admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:18.110468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887dab2d-9876-44b6-a650-4d758e4f373c admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:18.110857 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887dab2d-9876-44b6-a650-4d758e4f373c admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:18.111713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887dab2d-9876-44b6-a650-4d758e4f373c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:18.113861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887dab2d-9876-44b6-a650-4d758e4f373c admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:18.150012 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 939/1893] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:18 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:18.154197 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0332f-bc6a-49ae-950e-7969d9332806 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:18.240597 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0332f-bc6a-49ae-950e-7969d9332806 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:18.260827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:18.340634 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887dab2d-9876-44b6-a650-4d758e4f373c admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:18.344343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-887dab2d-9876-44b6-a650-4d758e4f373c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:18.359431 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49454f8f-cfbe-4b64-9d8b-198526f358d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:18.360435 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49454f8f-cfbe-4b64-9d8b-198526f358d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:18.361103 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49454f8f-cfbe-4b64-9d8b-198526f358d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:18.361359 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49454f8f-cfbe-4b64-9d8b-198526f358d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:18.363196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49454f8f-cfbe-4b64-9d8b-198526f358d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:18.365500 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49454f8f-cfbe-4b64-9d8b-198526f358d5 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:18.415592 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 955/1894] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:18 2022] POST /identity/v3/users => generated 289 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:18.419101 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49454f8f-cfbe-4b64-9d8b-198526f358d5 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:18.443007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49454f8f-cfbe-4b64-9d8b-198526f358d5 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:18.456225 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4df415-6ea1-4c8d-ad74-917b017f5223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:18.457007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4df415-6ea1-4c8d-ad74-917b017f5223 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:18.457319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4df415-6ea1-4c8d-ad74-917b017f5223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:18.459011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4df415-6ea1-4c8d-ad74-917b017f5223 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:18.462923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4df415-6ea1-4c8d-ad74-917b017f5223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:18.464631 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4df415-6ea1-4c8d-ad74-917b017f5223 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:18.479110 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 956/1895] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:18.483320 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4df415-6ea1-4c8d-ad74-917b017f5223 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:18.495646 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4df415-6ea1-4c8d-ad74-917b017f5223 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:18.502035 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738ed57-aefc-4f04-a2f0-cdf6fb6fe543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:18.511292 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738ed57-aefc-4f04-a2f0-cdf6fb6fe543 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:18.511292 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738ed57-aefc-4f04-a2f0-cdf6fb6fe543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:18.511292 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738ed57-aefc-4f04-a2f0-cdf6fb6fe543 admin admin] PATH_INFO: `/v3/projects/f5fd162694e6435bb003f05fc46390d4/users/750c131578cb415ea362cdef59d09407/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:18.514270 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887dab2d-9876-44b6-a650-4d758e4f373c admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:18.519138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb26e71-148c-4e46-9dc2-76381546a3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:18.519577 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738ed57-aefc-4f04-a2f0-cdf6fb6fe543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5fd162694e6435bb003f05fc46390d4, user_id=750c131578cb415ea362cdef59d09407, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:18.520151 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb26e71-148c-4e46-9dc2-76381546a3ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:18.520513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb26e71-148c-4e46-9dc2-76381546a3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:18.520890 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb26e71-148c-4e46-9dc2-76381546a3ad admin admin] PATH_INFO: `/v3/projects/8e7063e143a04ae98ab79f4e31ba525b` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:18.522154 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb26e71-148c-4e46-9dc2-76381546a3ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e7063e143a04ae98ab79f4e31ba525b)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:18.522551 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738ed57-aefc-4f04-a2f0-cdf6fb6fe543 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:18.526004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb26e71-148c-4e46-9dc2-76381546a3ad admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:18.537808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 957/1896] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:18 2022] PUT /identity/v3/projects/f5fd162694e6435bb003f05fc46390d4/users/750c131578cb415ea362cdef59d09407/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:18.542106 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738ed57-aefc-4f04-a2f0-cdf6fb6fe543 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:18.574107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-fdb26e71-148c-4e46-9dc2-76381546a3ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e7063e143a04ae98ab79f4e31ba525b', 'request_id': 'req-fdb26e71-148c-4e46-9dc2-76381546a3ad'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:18.580473 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-fdb26e71-148c-4e46-9dc2-76381546a3ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e7063e143a04ae98ab79f4e31ba525b', 'request_id': 'req-fdb26e71-148c-4e46-9dc2-76381546a3ad'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:18.589133 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-fdb26e71-148c-4e46-9dc2-76381546a3ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e7063e143a04ae98ab79f4e31ba525b', 'request_id': 'req-fdb26e71-148c-4e46-9dc2-76381546a3ad'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:18.600068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-fdb26e71-148c-4e46-9dc2-76381546a3ad admin admin] The token cache is being invalidate because project 8e7063e143a04ae98ab79f4e31ba525b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:18.600356 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-fdb26e71-148c-4e46-9dc2-76381546a3ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:18.601033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-fdb26e71-148c-4e46-9dc2-76381546a3ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:18.602655 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 940/1897] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:18 2022] DELETE /identity/v3/projects/8e7063e143a04ae98ab79f4e31ba525b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:18.606511 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb26e71-148c-4e46-9dc2-76381546a3ad admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:18.607288 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738ed57-aefc-4f04-a2f0-cdf6fb6fe543 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:18.607867 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-fdb26e71-148c-4e46-9dc2-76381546a3ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:18.612744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:18.668114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d30a78-58a8-4a64-9f54-111b4134ebbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:18.668856 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d30a78-58a8-4a64-9f54-111b4134ebbd admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:18.669145 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d30a78-58a8-4a64-9f54-111b4134ebbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:18.669422 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d30a78-58a8-4a64-9f54-111b4134ebbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:18.670153 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d30a78-58a8-4a64-9f54-111b4134ebbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:18.671756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d30a78-58a8-4a64-9f54-111b4134ebbd admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:18.674492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb26e71-148c-4e46-9dc2-76381546a3ad admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:18.679390 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e98ed-a7dc-425b-b4af-ec07326c9951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:18.680357 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e98ed-a7dc-425b-b4af-ec07326c9951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:18.680650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e98ed-a7dc-425b-b4af-ec07326c9951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:18.680965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e98ed-a7dc-425b-b4af-ec07326c9951 admin admin] PATH_INFO: `/v3/users/19820303a0a8459e88f8ec8069da1e6f` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:18.682235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e98ed-a7dc-425b-b4af-ec07326c9951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19820303a0a8459e88f8ec8069da1e6f)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:18.684822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 958/1898] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:18.684965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e98ed-a7dc-425b-b4af-ec07326c9951 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:18.686816 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d30a78-58a8-4a64-9f54-111b4134ebbd admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:18.688508 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e4d30a78-58a8-4a64-9f54-111b4134ebbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:18.702330 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d30a78-58a8-4a64-9f54-111b4134ebbd admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:18.707205 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b2916-efe8-487c-a86c-9f777f45009c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:18.708088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b2916-efe8-487c-a86c-9f777f45009c admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:18.708373 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b2916-efe8-487c-a86c-9f777f45009c admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:18.708653 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b2916-efe8-487c-a86c-9f777f45009c admin admin] PATH_INFO: `/v3/projects/f5fd162694e6435bb003f05fc46390d4/users/750c131578cb415ea362cdef59d09407/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:18.710832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b2916-efe8-487c-a86c-9f777f45009c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5fd162694e6435bb003f05fc46390d4, user_id=750c131578cb415ea362cdef59d09407, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:18.714725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b2916-efe8-487c-a86c-9f777f45009c admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:18.726825 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 959/1899] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:18 2022] PUT /identity/v3/projects/f5fd162694e6435bb003f05fc46390d4/users/750c131578cb415ea362cdef59d09407/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:18.735416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c95e98ed-a7dc-425b-b4af-ec07326c9951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19820303a0a8459e88f8ec8069da1e6f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:18.736850 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f8bde-3a3c-4650-9691-7175002ff784 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:18.737564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f8bde-3a3c-4650-9691-7175002ff784 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:18.737564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f8bde-3a3c-4650-9691-7175002ff784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:18.743217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:18.743645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c95e98ed-a7dc-425b-b4af-ec07326c9951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19820303a0a8459e88f8ec8069da1e6f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:18.752052 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c95e98ed-a7dc-425b-b4af-ec07326c9951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19820303a0a8459e88f8ec8069da1e6f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:18.752800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c95e98ed-a7dc-425b-b4af-ec07326c9951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19820303a0a8459e88f8ec8069da1e6f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:18.753353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c95e98ed-a7dc-425b-b4af-ec07326c9951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19820303a0a8459e88f8ec8069da1e6f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:18.768873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-176f8bde-3a3c-4650-9691-7175002ff784 None None] MFA Rules not processed for user `750c131578cb415ea362cdef59d09407`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:18.770109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-176f8bde-3a3c-4650-9691-7175002ff784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:18.771560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 941/1900] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:18 2022] DELETE /identity/v3/users/19820303a0a8459e88f8ec8069da1e6f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:18.847048 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 960/1901] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:18 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:18.859657 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e98ed-a7dc-425b-b4af-ec07326c9951 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:18.861131 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c95e98ed-a7dc-425b-b4af-ec07326c9951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:18.922691 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e98ed-a7dc-425b-b4af-ec07326c9951 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:18.926908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb44d0-2c04-41bf-af93-0d590a29775b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:18.927733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb44d0-2c04-41bf-af93-0d590a29775b service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:18.928004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb44d0-2c04-41bf-af93-0d590a29775b service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:18.928291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb44d0-2c04-41bf-af93-0d590a29775b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:18.938920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb44d0-2c04-41bf-af93-0d590a29775b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:18.940368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb44d0-2c04-41bf-af93-0d590a29775b service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:18.951710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 942/1902] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:18 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:19.048272 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f8bde-3a3c-4650-9691-7175002ff784 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:19.066750 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f8bde-3a3c-4650-9691-7175002ff784 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:19.072071 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6846043-06b0-4858-9e47-48ef38167b99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:19.072657 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6846043-06b0-4858-9e47-48ef38167b99 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:19.072976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6846043-06b0-4858-9e47-48ef38167b99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:19.073256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6846043-06b0-4858-9e47-48ef38167b99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:19.075153 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-f6846043-06b0-4858-9e47-48ef38167b99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:19.109703 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb44d0-2c04-41bf-af93-0d590a29775b service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:19.111218 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b3eb44d0-2c04-41bf-af93-0d590a29775b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:19.194054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:19.219617 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6846043-06b0-4858-9e47-48ef38167b99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:19.221997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6846043-06b0-4858-9e47-48ef38167b99 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:19.236219 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 961/1903] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:19 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:19.262106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb44d0-2c04-41bf-af93-0d590a29775b service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:19.268697 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129b973-c1ca-43bb-b2b8-56997947a2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:19.269666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129b973-c1ca-43bb-b2b8-56997947a2b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:19.270029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129b973-c1ca-43bb-b2b8-56997947a2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:19.270416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129b973-c1ca-43bb-b2b8-56997947a2b4 admin admin] PATH_INFO: `/v3/projects/f047bfce250d45e9b0a872cfdeb426a0` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:19.271968 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3129b973-c1ca-43bb-b2b8-56997947a2b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f047bfce250d45e9b0a872cfdeb426a0)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:19.275318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3129b973-c1ca-43bb-b2b8-56997947a2b4 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:19.366952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-3129b973-c1ca-43bb-b2b8-56997947a2b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f047bfce250d45e9b0a872cfdeb426a0', 'request_id': 'req-3129b973-c1ca-43bb-b2b8-56997947a2b4'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:19.375709 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-3129b973-c1ca-43bb-b2b8-56997947a2b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f047bfce250d45e9b0a872cfdeb426a0', 'request_id': 'req-3129b973-c1ca-43bb-b2b8-56997947a2b4'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:19.396349 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-3129b973-c1ca-43bb-b2b8-56997947a2b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f047bfce250d45e9b0a872cfdeb426a0', 'request_id': 'req-3129b973-c1ca-43bb-b2b8-56997947a2b4'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:19.413433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-3129b973-c1ca-43bb-b2b8-56997947a2b4 admin admin] The token cache is being invalidate because project f047bfce250d45e9b0a872cfdeb426a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:19.414678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-3129b973-c1ca-43bb-b2b8-56997947a2b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:19.415611 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-3129b973-c1ca-43bb-b2b8-56997947a2b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:19.417791 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 943/1904] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:19 2022] DELETE /identity/v3/projects/f047bfce250d45e9b0a872cfdeb426a0 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:19.424955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097169e-de07-4438-bf2a-ec6522696047 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:19.427162 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097169e-de07-4438-bf2a-ec6522696047 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:19.427889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097169e-de07-4438-bf2a-ec6522696047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:19.464199 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-a097169e-de07-4438-bf2a-ec6522696047 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:19.465515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a097169e-de07-4438-bf2a-ec6522696047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:19.625055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 962/1905] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:19.634305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23bac-c484-4d88-89a2-47313ad4b7ed None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:19.634798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23bac-c484-4d88-89a2-47313ad4b7ed None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:19.635175 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23bac-c484-4d88-89a2-47313ad4b7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:19.669215 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-5ff23bac-c484-4d88-89a2-47313ad4b7ed None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:19.670828 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-5ff23bac-c484-4d88-89a2-47313ad4b7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:19.703173 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 944/1906] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:19.707882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097169e-de07-4438-bf2a-ec6522696047 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:19.722725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097169e-de07-4438-bf2a-ec6522696047 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:19.731386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ef717-2a74-4482-aedf-1f9b173525e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:19.732995 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ef717-2a74-4482-aedf-1f9b173525e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:19.733560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ef717-2a74-4482-aedf-1f9b173525e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:19.735139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ef717-2a74-4482-aedf-1f9b173525e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:19.736214 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ef717-2a74-4482-aedf-1f9b173525e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:19.736803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ef717-2a74-4482-aedf-1f9b173525e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:19.739217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ef717-2a74-4482-aedf-1f9b173525e3 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:19.767179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 963/1907] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 22:06:19.771899 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff23bac-c484-4d88-89a2-47313ad4b7ed None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:19.790201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff23bac-c484-4d88-89a2-47313ad4b7ed None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:19.810470 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539da80-391d-4bee-a491-e8d20bb13e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:19.813780 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539da80-391d-4bee-a491-e8d20bb13e28 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:19.814361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539da80-391d-4bee-a491-e8d20bb13e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:19.814889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539da80-391d-4bee-a491-e8d20bb13e28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:19.815817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539da80-391d-4bee-a491-e8d20bb13e28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:19.818869 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539da80-391d-4bee-a491-e8d20bb13e28 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:19.880231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 945/1908] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:19 2022] POST /identity/v3/projects => generated 391 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:19.885276 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ef717-2a74-4482-aedf-1f9b173525e3 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:19.899190 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ef717-2a74-4482-aedf-1f9b173525e3 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:19.924135 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:20.057822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2866e-f7ac-401b-8c87-e913795ca594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:20.059018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2866e-f7ac-401b-8c87-e913795ca594 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:20.059665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2866e-f7ac-401b-8c87-e913795ca594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:20.060441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2866e-f7ac-401b-8c87-e913795ca594 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:20.061400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2866e-f7ac-401b-8c87-e913795ca594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:20.063650 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2866e-f7ac-401b-8c87-e913795ca594 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:20.102927 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 964/1909] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:19 2022] POST /identity/v3/users => generated 296 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:06:20.104329 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539da80-391d-4bee-a491-e8d20bb13e28 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:20.116157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539da80-391d-4bee-a491-e8d20bb13e28 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:20.121735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9851ac1-c8ef-4f01-831e-e93953c42e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:20.122635 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9851ac1-c8ef-4f01-831e-e93953c42e65 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:20.122993 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9851ac1-c8ef-4f01-831e-e93953c42e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:20.123359 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9851ac1-c8ef-4f01-831e-e93953c42e65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:20.124673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9851ac1-c8ef-4f01-831e-e93953c42e65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:20.128287 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9851ac1-c8ef-4f01-831e-e93953c42e65 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:20.153019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 946/1910] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:20.154555 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2866e-f7ac-401b-8c87-e913795ca594 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:20.178253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2866e-f7ac-401b-8c87-e913795ca594 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:20.192057 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88d456-d5cb-46eb-8f67-7d847ce1b47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:20.192963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88d456-d5cb-46eb-8f67-7d847ce1b47e admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:20.193255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88d456-d5cb-46eb-8f67-7d847ce1b47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:20.193654 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88d456-d5cb-46eb-8f67-7d847ce1b47e admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:20.196255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88d456-d5cb-46eb-8f67-7d847ce1b47e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:20.198346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88d456-d5cb-46eb-8f67-7d847ce1b47e admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:20.229868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 965/1911] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:06:20.231941 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9851ac1-c8ef-4f01-831e-e93953c42e65 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:20.251781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9851ac1-c8ef-4f01-831e-e93953c42e65 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:20.260999 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06c65c-92f8-4b9f-a49e-203b1f006961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:20.262829 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06c65c-92f8-4b9f-a49e-203b1f006961 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:20.263345 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06c65c-92f8-4b9f-a49e-203b1f006961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:20.263844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06c65c-92f8-4b9f-a49e-203b1f006961 admin admin] PATH_INFO: `/v3/projects/0e44b14612834f57a0f09f93badc97f2/users/0a23eaceb75f4d2ab052b3b805c442e0/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:20.294707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06c65c-92f8-4b9f-a49e-203b1f006961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e44b14612834f57a0f09f93badc97f2, user_id=0a23eaceb75f4d2ab052b3b805c442e0, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:20.297931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06c65c-92f8-4b9f-a49e-203b1f006961 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:20.321223 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 947/1912] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:20 2022] PUT /identity/v3/projects/0e44b14612834f57a0f09f93badc97f2/users/0a23eaceb75f4d2ab052b3b805c442e0/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:20.333181 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a480b71-3fc0-40cf-b313-e75cef79a994 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:20.333487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a480b71-3fc0-40cf-b313-e75cef79a994 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:20.333813 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a480b71-3fc0-40cf-b313-e75cef79a994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:20.376460 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-1a480b71-3fc0-40cf-b313-e75cef79a994 None None] MFA Rules not processed for user `0a23eaceb75f4d2ab052b3b805c442e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:20.377897 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-1a480b71-3fc0-40cf-b313-e75cef79a994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:20.508786 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 966/1913] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:20.512210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06c65c-92f8-4b9f-a49e-203b1f006961 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:20.596890 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:20.618019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06c65c-92f8-4b9f-a49e-203b1f006961 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:20.633458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75c8f5-7b92-43eb-ae57-2baf76c64f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:20.634152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75c8f5-7b92-43eb-ae57-2baf76c64f4c admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:20.634637 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75c8f5-7b92-43eb-ae57-2baf76c64f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:20.634762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75c8f5-7b92-43eb-ae57-2baf76c64f4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:20.635393 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75c8f5-7b92-43eb-ae57-2baf76c64f4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:20.637650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75c8f5-7b92-43eb-ae57-2baf76c64f4c admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:20.684761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 948/1914] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:20 2022] POST /identity/v3/projects => generated 391 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:06:20.685646 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a480b71-3fc0-40cf-b313-e75cef79a994 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:20.709141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a480b71-3fc0-40cf-b313-e75cef79a994 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:20.798477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8928663-f767-402a-ada1-b4bbafbd83c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:20.799332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8928663-f767-402a-ada1-b4bbafbd83c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:20.799622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8928663-f767-402a-ada1-b4bbafbd83c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:20.799902 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8928663-f767-402a-ada1-b4bbafbd83c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:20.800604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8928663-f767-402a-ada1-b4bbafbd83c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:20.802590 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8928663-f767-402a-ada1-b4bbafbd83c7 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:20.831324 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 967/1915] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:20 2022] POST /identity/v3/users => generated 302 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:20.832652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75c8f5-7b92-43eb-ae57-2baf76c64f4c admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:20.853946 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75c8f5-7b92-43eb-ae57-2baf76c64f4c admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:20.863960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271430c-7f5f-41d0-aee8-dddd674be798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:20.864653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271430c-7f5f-41d0-aee8-dddd674be798 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:20.864969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271430c-7f5f-41d0-aee8-dddd674be798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:20.865256 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271430c-7f5f-41d0-aee8-dddd674be798 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:20.866190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271430c-7f5f-41d0-aee8-dddd674be798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:20.868394 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271430c-7f5f-41d0-aee8-dddd674be798 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:20.882808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 949/1916] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:06:20.888938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8928663-f767-402a-ada1-b4bbafbd83c7 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:20.901880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8928663-f767-402a-ada1-b4bbafbd83c7 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:20.907524 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a5353-4956-4854-a332-cb425a127c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:20.908794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a5353-4956-4854-a332-cb425a127c02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:20.909154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a5353-4956-4854-a332-cb425a127c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:20.909508 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a5353-4956-4854-a332-cb425a127c02 admin admin] PATH_INFO: `/v3/projects/51399ff174e34f2d8937c9a7197d837f/users/ae5160af6d234c5da244f51521499ba9/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:20.920682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:20.937580 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a5353-4956-4854-a332-cb425a127c02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51399ff174e34f2d8937c9a7197d837f, user_id=ae5160af6d234c5da244f51521499ba9, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:20.940347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a5353-4956-4854-a332-cb425a127c02 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:20.970399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 968/1917] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:20 2022] PUT /identity/v3/projects/51399ff174e34f2d8937c9a7197d837f/users/ae5160af6d234c5da244f51521499ba9/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:20.973621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271430c-7f5f-41d0-aee8-dddd674be798 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:21.000644 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271430c-7f5f-41d0-aee8-dddd674be798 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:21.107752 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d7517-011d-4be3-846e-51b2be493e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:21.113869 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d7517-011d-4be3-846e-51b2be493e1e admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:21.114376 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d7517-011d-4be3-846e-51b2be493e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:21.114743 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d7517-011d-4be3-846e-51b2be493e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:21.116506 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d7517-011d-4be3-846e-51b2be493e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:21.118401 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d7517-011d-4be3-846e-51b2be493e1e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:21.136737 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 950/1918] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:21.140800 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a5353-4956-4854-a332-cb425a127c02 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:21.155139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a5353-4956-4854-a332-cb425a127c02 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:21.168967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e1014-e4ed-4c27-a6e2-25481b5bbebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:21.168967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e1014-e4ed-4c27-a6e2-25481b5bbebc admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:21.168967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e1014-e4ed-4c27-a6e2-25481b5bbebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:21.169992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e1014-e4ed-4c27-a6e2-25481b5bbebc admin admin] PATH_INFO: `/v3/projects/51399ff174e34f2d8937c9a7197d837f/users/ae5160af6d234c5da244f51521499ba9/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:21.175773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e1014-e4ed-4c27-a6e2-25481b5bbebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=51399ff174e34f2d8937c9a7197d837f, user_id=ae5160af6d234c5da244f51521499ba9, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:21.182447 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e1014-e4ed-4c27-a6e2-25481b5bbebc admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:21.201616 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 969/1919] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:21 2022] PUT /identity/v3/projects/51399ff174e34f2d8937c9a7197d837f/users/ae5160af6d234c5da244f51521499ba9/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:21.213984 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb43d6-054a-4f56-924a-bbd7a5cdf614 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:21.214613 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb43d6-054a-4f56-924a-bbd7a5cdf614 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:21.215118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb43d6-054a-4f56-924a-bbd7a5cdf614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:21.271990 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-ccfb43d6-054a-4f56-924a-bbd7a5cdf614 None None] MFA Rules not processed for user `ae5160af6d234c5da244f51521499ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:21.274238 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ccfb43d6-054a-4f56-924a-bbd7a5cdf614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:21.368224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 951/1920] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:21 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:21.379596 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e1014-e4ed-4c27-a6e2-25481b5bbebc admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:21.381609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-285e1014-e4ed-4c27-a6e2-25481b5bbebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:21.460449 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e1014-e4ed-4c27-a6e2-25481b5bbebc admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:21.464984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109cf801-d963-491f-9f6d-c7ac8ea70195 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:21.465724 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109cf801-d963-491f-9f6d-c7ac8ea70195 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:21.465938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109cf801-d963-491f-9f6d-c7ac8ea70195 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:21.466222 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109cf801-d963-491f-9f6d-c7ac8ea70195 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:21.477130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109cf801-d963-491f-9f6d-c7ac8ea70195 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:21.478607 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109cf801-d963-491f-9f6d-c7ac8ea70195 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:21.489991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 970/1921] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:21.554049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb43d6-054a-4f56-924a-bbd7a5cdf614 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:21.558197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ccfb43d6-054a-4f56-924a-bbd7a5cdf614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:21.575225 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb43d6-054a-4f56-924a-bbd7a5cdf614 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:21.579164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d93033-b553-4f97-8254-18ff109d8daf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:21.579833 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d93033-b553-4f97-8254-18ff109d8daf service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:21.580122 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d93033-b553-4f97-8254-18ff109d8daf service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:21.580402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d93033-b553-4f97-8254-18ff109d8daf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:21.640920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:21.673359 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d93033-b553-4f97-8254-18ff109d8daf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:21.674854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d93033-b553-4f97-8254-18ff109d8daf service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:21.685019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 952/1922] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:25.564400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109cf801-d963-491f-9f6d-c7ac8ea70195 service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:25.566873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-109cf801-d963-491f-9f6d-c7ac8ea70195 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:25.697776 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109cf801-d963-491f-9f6d-c7ac8ea70195 service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:25.703369 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5040bb-c973-4856-abf4-01471deb8a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:25.704570 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5040bb-c973-4856-abf4-01471deb8a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:25.704986 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5040bb-c973-4856-abf4-01471deb8a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:25.705388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5040bb-c973-4856-abf4-01471deb8a5f admin admin] PATH_INFO: `/v3/users/e5a3f9744cdf409ca2800cf2d7a799c2` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:25.709610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5040bb-c973-4856-abf4-01471deb8a5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5a3f9744cdf409ca2800cf2d7a799c2)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:25.709610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5040bb-c973-4856-abf4-01471deb8a5f admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:25.813402 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3b5040bb-c973-4856-abf4-01471deb8a5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5a3f9744cdf409ca2800cf2d7a799c2'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:25.854653 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3b5040bb-c973-4856-abf4-01471deb8a5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5a3f9744cdf409ca2800cf2d7a799c2'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:25.884038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3b5040bb-c973-4856-abf4-01471deb8a5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5a3f9744cdf409ca2800cf2d7a799c2'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:25.892550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:25.900180 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3b5040bb-c973-4856-abf4-01471deb8a5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5a3f9744cdf409ca2800cf2d7a799c2'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:25.901499 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3b5040bb-c973-4856-abf4-01471deb8a5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5a3f9744cdf409ca2800cf2d7a799c2'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:25.915169 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 971/1923] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:25 2022] DELETE /identity/v3/users/e5a3f9744cdf409ca2800cf2d7a799c2 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:25.920833 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e8467-45af-40da-82cf-941e527f23d1 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:25.920833 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e8467-45af-40da-82cf-941e527f23d1 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:25.920833 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e8467-45af-40da-82cf-941e527f23d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:25.951825 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-1c4e8467-45af-40da-82cf-941e527f23d1 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:25.953638 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1c4e8467-45af-40da-82cf-941e527f23d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:25.984018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5040bb-c973-4856-abf4-01471deb8a5f admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:25.985557 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-3b5040bb-c973-4856-abf4-01471deb8a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:26.127288 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 953/1924] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:26.144508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e8467-45af-40da-82cf-941e527f23d1 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:26.146021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1c4e8467-45af-40da-82cf-941e527f23d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:26.171234 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5040bb-c973-4856-abf4-01471deb8a5f admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:26.186436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d9772-ed4f-464a-9c1c-0166776bb600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:26.187409 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d9772-ed4f-464a-9c1c-0166776bb600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:26.189067 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d9772-ed4f-464a-9c1c-0166776bb600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:26.189544 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d9772-ed4f-464a-9c1c-0166776bb600 admin admin] PATH_INFO: `/v3/users/0a23eaceb75f4d2ab052b3b805c442e0` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:26.197653 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d9772-ed4f-464a-9c1c-0166776bb600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a23eaceb75f4d2ab052b3b805c442e0)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:26.200007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d9772-ed4f-464a-9c1c-0166776bb600 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:26.227088 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e8467-45af-40da-82cf-941e527f23d1 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:26.232114 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf684ce-cbbc-4256-8db7-ddc05ee8f456 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:26.233832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf684ce-cbbc-4256-8db7-ddc05ee8f456 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:26.234352 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf684ce-cbbc-4256-8db7-ddc05ee8f456 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:26.234998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf684ce-cbbc-4256-8db7-ddc05ee8f456 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:26.250906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf684ce-cbbc-4256-8db7-ddc05ee8f456 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:26.254281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf684ce-cbbc-4256-8db7-ddc05ee8f456 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:26.271458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 954/1925] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:06:26.312863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ae6d9772-ed4f-464a-9c1c-0166776bb600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a23eaceb75f4d2ab052b3b805c442e0'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:26.349784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ae6d9772-ed4f-464a-9c1c-0166776bb600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a23eaceb75f4d2ab052b3b805c442e0'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:26.375174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ae6d9772-ed4f-464a-9c1c-0166776bb600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a23eaceb75f4d2ab052b3b805c442e0'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:26.386571 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ae6d9772-ed4f-464a-9c1c-0166776bb600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a23eaceb75f4d2ab052b3b805c442e0'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:26.388782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ae6d9772-ed4f-464a-9c1c-0166776bb600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a23eaceb75f4d2ab052b3b805c442e0'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:26.400804 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 972/1926] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:25 2022] DELETE /identity/v3/users/0a23eaceb75f4d2ab052b3b805c442e0 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:26.405716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38588bdb-9f62-453e-89b8-e7576c381f6b None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:26.406199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38588bdb-9f62-453e-89b8-e7576c381f6b None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:26.406633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38588bdb-9f62-453e-89b8-e7576c381f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:26.435259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-38588bdb-9f62-453e-89b8-e7576c381f6b None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:26.436552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-38588bdb-9f62-453e-89b8-e7576c381f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:26.555247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 955/1927] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:26.565265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d9772-ed4f-464a-9c1c-0166776bb600 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:26.567015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ae6d9772-ed4f-464a-9c1c-0166776bb600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:26.675477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d9772-ed4f-464a-9c1c-0166776bb600 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:26.681745 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edf075-51cd-455d-a160-33c8979690be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:26.682663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edf075-51cd-455d-a160-33c8979690be service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:26.682663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edf075-51cd-455d-a160-33c8979690be service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:26.683000 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edf075-51cd-455d-a160-33c8979690be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:26.696139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edf075-51cd-455d-a160-33c8979690be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:26.697742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edf075-51cd-455d-a160-33c8979690be service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:26.710988 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 973/1928] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:26.976684 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38588bdb-9f62-453e-89b8-e7576c381f6b None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:26.979645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-38588bdb-9f62-453e-89b8-e7576c381f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:26.994411 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38588bdb-9f62-453e-89b8-e7576c381f6b None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:27.001510 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99266099-679e-48c6-8a7d-4b7fbfd9e496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:27.002483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99266099-679e-48c6-8a7d-4b7fbfd9e496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:27.002842 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99266099-679e-48c6-8a7d-4b7fbfd9e496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:27.003318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99266099-679e-48c6-8a7d-4b7fbfd9e496 admin admin] PATH_INFO: `/v3/projects/6c4e4a6f43d04d5d9cd3bea2ecea18c8` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:27.004733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99266099-679e-48c6-8a7d-4b7fbfd9e496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c4e4a6f43d04d5d9cd3bea2ecea18c8)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:27.006703 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99266099-679e-48c6-8a7d-4b7fbfd9e496 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:27.033875 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:27.071910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-99266099-679e-48c6-8a7d-4b7fbfd9e496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c4e4a6f43d04d5d9cd3bea2ecea18c8', 'request_id': 'req-99266099-679e-48c6-8a7d-4b7fbfd9e496'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:27.088882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-99266099-679e-48c6-8a7d-4b7fbfd9e496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c4e4a6f43d04d5d9cd3bea2ecea18c8', 'request_id': 'req-99266099-679e-48c6-8a7d-4b7fbfd9e496'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:27.100173 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-99266099-679e-48c6-8a7d-4b7fbfd9e496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c4e4a6f43d04d5d9cd3bea2ecea18c8', 'request_id': 'req-99266099-679e-48c6-8a7d-4b7fbfd9e496'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:27.122787 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-99266099-679e-48c6-8a7d-4b7fbfd9e496 admin admin] The token cache is being invalidate because project 6c4e4a6f43d04d5d9cd3bea2ecea18c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:27.123218 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-99266099-679e-48c6-8a7d-4b7fbfd9e496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:27.124316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-99266099-679e-48c6-8a7d-4b7fbfd9e496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:27.126085 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 956/1929] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:26 2022] DELETE /identity/v3/projects/6c4e4a6f43d04d5d9cd3bea2ecea18c8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:27.130363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edf075-51cd-455d-a160-33c8979690be service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:27.145857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edf075-51cd-455d-a160-33c8979690be service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:27.170008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:27.231927 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb15a1-41fd-410b-97ae-20936e8e7af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:27.233112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb15a1-41fd-410b-97ae-20936e8e7af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:27.233472 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb15a1-41fd-410b-97ae-20936e8e7af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:27.233861 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb15a1-41fd-410b-97ae-20936e8e7af9 admin admin] PATH_INFO: `/v3/users/442e2b7f02a24ff69f61ae94500c2e5f` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:27.235671 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb15a1-41fd-410b-97ae-20936e8e7af9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=442e2b7f02a24ff69f61ae94500c2e5f)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:27.238157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb15a1-41fd-410b-97ae-20936e8e7af9 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:27.317025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-aebb15a1-41fd-410b-97ae-20936e8e7af9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '442e2b7f02a24ff69f61ae94500c2e5f'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:27.339138 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-aebb15a1-41fd-410b-97ae-20936e8e7af9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '442e2b7f02a24ff69f61ae94500c2e5f'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:27.346822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-aebb15a1-41fd-410b-97ae-20936e8e7af9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '442e2b7f02a24ff69f61ae94500c2e5f'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:27.357014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-aebb15a1-41fd-410b-97ae-20936e8e7af9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '442e2b7f02a24ff69f61ae94500c2e5f'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:27.357958 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-aebb15a1-41fd-410b-97ae-20936e8e7af9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '442e2b7f02a24ff69f61ae94500c2e5f'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:27.366218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 974/1930] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:27 2022] DELETE /identity/v3/users/442e2b7f02a24ff69f61ae94500c2e5f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:27.412466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e20298-be5f-42bb-89b7-a958bf01b925 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:27.414865 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e20298-be5f-42bb-89b7-a958bf01b925 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:27.415538 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e20298-be5f-42bb-89b7-a958bf01b925 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:27.418346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 957/1931] 10.222.0.44 () {54 vars in 921 bytes} [Tue Jul 12 22:06:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:06:27.428095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c980e1-03b7-4fdb-b0d8-6a8f0c6a9464 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:27.428375 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c980e1-03b7-4fdb-b0d8-6a8f0c6a9464 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:27.428667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c980e1-03b7-4fdb-b0d8-6a8f0c6a9464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:27.448464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-78c980e1-03b7-4fdb-b0d8-6a8f0c6a9464 None None] MFA Rules not processed for user `24915625354f418eb7bc4d5ada85fdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:27.449758 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-78c980e1-03b7-4fdb-b0d8-6a8f0c6a9464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:27.543187 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 975/1932] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Jul 12 22:06:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:27.545512 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e20298-be5f-42bb-89b7-a958bf01b925 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:27.547848 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e4e20298-be5f-42bb-89b7-a958bf01b925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:27.560755 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c980e1-03b7-4fdb-b0d8-6a8f0c6a9464 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:27.563828 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-78c980e1-03b7-4fdb-b0d8-6a8f0c6a9464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:27.590702 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c980e1-03b7-4fdb-b0d8-6a8f0c6a9464 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:27.599693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d4323-aca9-4889-8129-c6aa6a9e4b06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:27.601001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d4323-aca9-4889-8129-c6aa6a9e4b06 service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:27.601617 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d4323-aca9-4889-8129-c6aa6a9e4b06 service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:27.602310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d4323-aca9-4889-8129-c6aa6a9e4b06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:27.621091 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d4323-aca9-4889-8129-c6aa6a9e4b06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:27.623240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d4323-aca9-4889-8129-c6aa6a9e4b06 service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:27.641164 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 976/1933] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:27.705466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e20298-be5f-42bb-89b7-a958bf01b925 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:27.725983 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:27.727994 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:27.728501 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:27.729003 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb admin admin] PATH_INFO: `/v3/projects/0e44b14612834f57a0f09f93badc97f2` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:27.732150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e44b14612834f57a0f09f93badc97f2)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:27.736086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:27.851795 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e44b14612834f57a0f09f93badc97f2', 'request_id': 'req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:27.865901 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e44b14612834f57a0f09f93badc97f2', 'request_id': 'req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:27.892464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e44b14612834f57a0f09f93badc97f2', 'request_id': 'req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:27.914604 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb admin admin] The token cache is being invalidate because project 0e44b14612834f57a0f09f93badc97f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:27.914942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:27.916146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:27.926147 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 958/1934] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:27 2022] DELETE /identity/v3/projects/0e44b14612834f57a0f09f93badc97f2 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:27.927885 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d4323-aca9-4889-8129-c6aa6a9e4b06 service nova] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:27.942150 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d4323-aca9-4889-8129-c6aa6a9e4b06 service nova] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:27.969549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:28.104386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05f7aa-34ad-4253-9938-715e5cbe7e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:28.105509 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05f7aa-34ad-4253-9938-715e5cbe7e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:28.107856 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05f7aa-34ad-4253-9938-715e5cbe7e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:28.108320 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05f7aa-34ad-4253-9938-715e5cbe7e1a admin admin] PATH_INFO: `/v3/users/ae5160af6d234c5da244f51521499ba9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:28.110374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd05f7aa-34ad-4253-9938-715e5cbe7e1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae5160af6d234c5da244f51521499ba9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:28.116625 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd05f7aa-34ad-4253-9938-715e5cbe7e1a admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:28.235463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dd05f7aa-34ad-4253-9938-715e5cbe7e1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae5160af6d234c5da244f51521499ba9'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:28.276236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:28.277765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:28.298800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf610d5-25ee-4ee1-afe3-3cdbaaea6bfb admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:28.303849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dd05f7aa-34ad-4253-9938-715e5cbe7e1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae5160af6d234c5da244f51521499ba9'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:28.310432 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2f6ba-68e7-494d-be9b-f4d3cc76b369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:28.311720 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2f6ba-68e7-494d-be9b-f4d3cc76b369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:28.312162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2f6ba-68e7-494d-be9b-f4d3cc76b369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:28.313908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2f6ba-68e7-494d-be9b-f4d3cc76b369 admin admin] PATH_INFO: `/v3/users/b58340f1f571471fbb535968cc4f1786` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:28.316422 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2f6ba-68e7-494d-be9b-f4d3cc76b369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b58340f1f571471fbb535968cc4f1786)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:28.318563 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dd05f7aa-34ad-4253-9938-715e5cbe7e1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae5160af6d234c5da244f51521499ba9'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:28.324490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2f6ba-68e7-494d-be9b-f4d3cc76b369 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:28.337456 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dd05f7aa-34ad-4253-9938-715e5cbe7e1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae5160af6d234c5da244f51521499ba9'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:28.338218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dd05f7aa-34ad-4253-9938-715e5cbe7e1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae5160af6d234c5da244f51521499ba9'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:28.351916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 977/1935] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:27 2022] DELETE /identity/v3/users/ae5160af6d234c5da244f51521499ba9 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:28.358029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05f7aa-34ad-4253-9938-715e5cbe7e1a admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:28.361985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-dd05f7aa-34ad-4253-9938-715e5cbe7e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:28.405333 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:28.426780 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0ad2f6ba-68e7-494d-be9b-f4d3cc76b369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b58340f1f571471fbb535968cc4f1786'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:28.444027 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0ad2f6ba-68e7-494d-be9b-f4d3cc76b369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b58340f1f571471fbb535968cc4f1786'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:28.458977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0ad2f6ba-68e7-494d-be9b-f4d3cc76b369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b58340f1f571471fbb535968cc4f1786'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:28.460107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0ad2f6ba-68e7-494d-be9b-f4d3cc76b369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b58340f1f571471fbb535968cc4f1786'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:28.463270 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0ad2f6ba-68e7-494d-be9b-f4d3cc76b369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b58340f1f571471fbb535968cc4f1786'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:28.503949 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 959/1936] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:28 2022] DELETE /identity/v3/users/b58340f1f571471fbb535968cc4f1786 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:28.509347 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42ce0e-b43b-4c0d-aca0-e7bb31382deb None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:28.509904 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42ce0e-b43b-4c0d-aca0-e7bb31382deb None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:28.511209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42ce0e-b43b-4c0d-aca0-e7bb31382deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:28.531278 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05f7aa-34ad-4253-9938-715e5cbe7e1a admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:28.570656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-4a42ce0e-b43b-4c0d-aca0-e7bb31382deb None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:28.572388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4a42ce0e-b43b-4c0d-aca0-e7bb31382deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:28.628200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:28.671893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628da89-036a-42f7-aa71-0f923f1e7583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:28.672880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628da89-036a-42f7-aa71-0f923f1e7583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:28.673070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628da89-036a-42f7-aa71-0f923f1e7583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:28.673370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628da89-036a-42f7-aa71-0f923f1e7583 admin admin] PATH_INFO: `/v3/projects/e33ab7281bc34df9a35f2e7fe73d5b5e` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:28.677587 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628da89-036a-42f7-aa71-0f923f1e7583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e33ab7281bc34df9a35f2e7fe73d5b5e)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:28.679761 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628da89-036a-42f7-aa71-0f923f1e7583 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:28.745806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 960/1937] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:28.770643 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42ce0e-b43b-4c0d-aca0-e7bb31382deb None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:28.780690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4a42ce0e-b43b-4c0d-aca0-e7bb31382deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:28.796171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-a628da89-036a-42f7-aa71-0f923f1e7583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e33ab7281bc34df9a35f2e7fe73d5b5e', 'request_id': 'req-a628da89-036a-42f7-aa71-0f923f1e7583'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:28.817847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-a628da89-036a-42f7-aa71-0f923f1e7583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e33ab7281bc34df9a35f2e7fe73d5b5e', 'request_id': 'req-a628da89-036a-42f7-aa71-0f923f1e7583'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:28.834003 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-a628da89-036a-42f7-aa71-0f923f1e7583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e33ab7281bc34df9a35f2e7fe73d5b5e', 'request_id': 'req-a628da89-036a-42f7-aa71-0f923f1e7583'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:28.857386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-a628da89-036a-42f7-aa71-0f923f1e7583 admin admin] The token cache is being invalidate because project e33ab7281bc34df9a35f2e7fe73d5b5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:28.857926 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-a628da89-036a-42f7-aa71-0f923f1e7583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:28.860851 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-a628da89-036a-42f7-aa71-0f923f1e7583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:28.864379 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 978/1938] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:28 2022] DELETE /identity/v3/projects/e33ab7281bc34df9a35f2e7fe73d5b5e => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:28.871322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031efef-899c-4e5b-9d15-f1d5b6222484 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:28.871322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031efef-899c-4e5b-9d15-f1d5b6222484 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:28.871610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031efef-899c-4e5b-9d15-f1d5b6222484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:28.917883 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-9031efef-899c-4e5b-9d15-f1d5b6222484 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:28.921277 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-9031efef-899c-4e5b-9d15-f1d5b6222484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:28.927419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42ce0e-b43b-4c0d-aca0-e7bb31382deb None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:29.001521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d25fd4-f5e2-46f8-99a5-fbac68414f23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:29.002367 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d25fd4-f5e2-46f8-99a5-fbac68414f23 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:29.002853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d25fd4-f5e2-46f8-99a5-fbac68414f23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:29.003257 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d25fd4-f5e2-46f8-99a5-fbac68414f23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:29.006900 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c6d25fd4-f5e2-46f8-99a5-fbac68414f23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:29.024485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 979/1939] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:29.026153 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c2a7e-2e33-43dd-bd2c-85c83f7b1f0c None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:29.026307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d25fd4-f5e2-46f8-99a5-fbac68414f23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:29.026403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c2a7e-2e33-43dd-bd2c-85c83f7b1f0c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:29.026744 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c2a7e-2e33-43dd-bd2c-85c83f7b1f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:29.031163 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d25fd4-f5e2-46f8-99a5-fbac68414f23 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:29.047951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 961/1940] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:29.074395 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-f31c2a7e-2e33-43dd-bd2c-85c83f7b1f0c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:29.078094 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-f31c2a7e-2e33-43dd-bd2c-85c83f7b1f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:29.095186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 980/1941] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:29.098237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d25fd4-f5e2-46f8-99a5-fbac68414f23 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:29.110782 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d25fd4-f5e2-46f8-99a5-fbac68414f23 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:29.123438 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e668b-a463-4763-b5b0-41b27d2d78c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:29.124198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e668b-a463-4763-b5b0-41b27d2d78c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:29.124485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e668b-a463-4763-b5b0-41b27d2d78c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:29.124906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e668b-a463-4763-b5b0-41b27d2d78c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:29.126217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e668b-a463-4763-b5b0-41b27d2d78c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:29.126646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e668b-a463-4763-b5b0-41b27d2d78c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:29.128564 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e668b-a463-4763-b5b0-41b27d2d78c5 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:29.139916 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:29.145861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 962/1942] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:06:29.150246 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c2a7e-2e33-43dd-bd2c-85c83f7b1f0c None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:29.161856 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c2a7e-2e33-43dd-bd2c-85c83f7b1f0c None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:29.178217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e62e5f-6737-433e-89f6-167251659366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:29.179103 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e62e5f-6737-433e-89f6-167251659366 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:29.179484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e62e5f-6737-433e-89f6-167251659366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:29.179768 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e62e5f-6737-433e-89f6-167251659366 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:29.180588 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e62e5f-6737-433e-89f6-167251659366 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:29.182779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e62e5f-6737-433e-89f6-167251659366 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:29.241051 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 981/1943] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:29 2022] POST /identity/v3/projects => generated 379 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:06:29.242559 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e668b-a463-4763-b5b0-41b27d2d78c5 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:29.258580 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e668b-a463-4763-b5b0-41b27d2d78c5 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:29.357732 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4963d1-685d-4655-a479-cca1c41b7603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:29.358583 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4963d1-685d-4655-a479-cca1c41b7603 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:29.358970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4963d1-685d-4655-a479-cca1c41b7603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:29.359349 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4963d1-685d-4655-a479-cca1c41b7603 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:29.360182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4963d1-685d-4655-a479-cca1c41b7603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:29.362411 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4963d1-685d-4655-a479-cca1c41b7603 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:29.393485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 963/1944] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:29 2022] POST /identity/v3/users => generated 290 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:29.394505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e62e5f-6737-433e-89f6-167251659366 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:29.406404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e62e5f-6737-433e-89f6-167251659366 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:29.415351 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0ffbb-2c91-4f4a-bbf4-cb6f403bb5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:29.416037 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0ffbb-2c91-4f4a-bbf4-cb6f403bb5cb admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:29.416328 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0ffbb-2c91-4f4a-bbf4-cb6f403bb5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:29.416680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0ffbb-2c91-4f4a-bbf4-cb6f403bb5cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:29.417460 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0ffbb-2c91-4f4a-bbf4-cb6f403bb5cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:29.419317 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0ffbb-2c91-4f4a-bbf4-cb6f403bb5cb admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:29.431123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 982/1945] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:29.435248 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4963d1-685d-4655-a479-cca1c41b7603 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:29.453384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4963d1-685d-4655-a479-cca1c41b7603 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:29.459189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7ffd2-c5b9-4378-8cad-44774f8837cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:29.459881 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7ffd2-c5b9-4378-8cad-44774f8837cb admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:29.460210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7ffd2-c5b9-4378-8cad-44774f8837cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:29.461374 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7ffd2-c5b9-4378-8cad-44774f8837cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:29.462237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d7ffd2-c5b9-4378-8cad-44774f8837cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:29.464285 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d7ffd2-c5b9-4378-8cad-44774f8837cb admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:29.478633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 964/1946] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:29.479909 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0ffbb-2c91-4f4a-bbf4-cb6f403bb5cb admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:29.490859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0ffbb-2c91-4f4a-bbf4-cb6f403bb5cb admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:29.496732 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19521e5c-b8ff-4a0a-9973-b7c25cceb1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:29.497819 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19521e5c-b8ff-4a0a-9973-b7c25cceb1ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:29.498084 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19521e5c-b8ff-4a0a-9973-b7c25cceb1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:29.498438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19521e5c-b8ff-4a0a-9973-b7c25cceb1ad admin admin] PATH_INFO: `/v3/projects/2e27bde984bc443dbb465165f0a79b28/users/507b217f9e104f6f9c9d80541695b423/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:29.520442 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19521e5c-b8ff-4a0a-9973-b7c25cceb1ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e27bde984bc443dbb465165f0a79b28, user_id=507b217f9e104f6f9c9d80541695b423, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:29.524217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19521e5c-b8ff-4a0a-9973-b7c25cceb1ad admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:29.549846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 983/1947] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:29 2022] PUT /identity/v3/projects/2e27bde984bc443dbb465165f0a79b28/users/507b217f9e104f6f9c9d80541695b423/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:29.563750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3a3b4-c722-45f1-938e-1610f6518e76 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:29.564207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3a3b4-c722-45f1-938e-1610f6518e76 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:29.565163 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3a3b4-c722-45f1-938e-1610f6518e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:29.609054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-97c3a3b4-c722-45f1-938e-1610f6518e76 None None] MFA Rules not processed for user `507b217f9e104f6f9c9d80541695b423`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:29.610428 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-97c3a3b4-c722-45f1-938e-1610f6518e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:29.649676 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19521e5c-b8ff-4a0a-9973-b7c25cceb1ad admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:29.651581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-19521e5c-b8ff-4a0a-9973-b7c25cceb1ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:29.667741 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:29.734451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 965/1948] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:29.739074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3a3b4-c722-45f1-938e-1610f6518e76 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:29.781801 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19521e5c-b8ff-4a0a-9973-b7c25cceb1ad admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:29.788439 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b28a3-b9e3-4285-8239-3c72e95400bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:29.790155 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b28a3-b9e3-4285-8239-3c72e95400bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:29.790510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b28a3-b9e3-4285-8239-3c72e95400bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:29.790848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b28a3-b9e3-4285-8239-3c72e95400bf admin admin] PATH_INFO: `/v3/projects/51399ff174e34f2d8937c9a7197d837f` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:29.792938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b28a3-b9e3-4285-8239-3c72e95400bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=51399ff174e34f2d8937c9a7197d837f)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:29.794860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b28a3-b9e3-4285-8239-3c72e95400bf admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:29.843720 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3a3b4-c722-45f1-938e-1610f6518e76 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:29.850051 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96dcc46-47ef-43c3-b127-b5ebec3f8000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:29.850797 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96dcc46-47ef-43c3-b127-b5ebec3f8000 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:29.851685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96dcc46-47ef-43c3-b127-b5ebec3f8000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:29.852729 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96dcc46-47ef-43c3-b127-b5ebec3f8000 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:29.853534 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96dcc46-47ef-43c3-b127-b5ebec3f8000 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:29.855537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96dcc46-47ef-43c3-b127-b5ebec3f8000 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:29.870514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-019b28a3-b9e3-4285-8239-3c72e95400bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51399ff174e34f2d8937c9a7197d837f', 'request_id': 'req-019b28a3-b9e3-4285-8239-3c72e95400bf'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:29.875140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:29.878622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-019b28a3-b9e3-4285-8239-3c72e95400bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51399ff174e34f2d8937c9a7197d837f', 'request_id': 'req-019b28a3-b9e3-4285-8239-3c72e95400bf'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:29.893093 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-019b28a3-b9e3-4285-8239-3c72e95400bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51399ff174e34f2d8937c9a7197d837f', 'request_id': 'req-019b28a3-b9e3-4285-8239-3c72e95400bf'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:29.898209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 966/1949] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:29 2022] POST /identity/v3/projects => generated 379 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:29.899488 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96dcc46-47ef-43c3-b127-b5ebec3f8000 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:29.901436 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b96dcc46-47ef-43c3-b127-b5ebec3f8000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:29.906661 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-019b28a3-b9e3-4285-8239-3c72e95400bf admin admin] The token cache is being invalidate because project 51399ff174e34f2d8937c9a7197d837f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:29.906946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-019b28a3-b9e3-4285-8239-3c72e95400bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:29.907689 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-019b28a3-b9e3-4285-8239-3c72e95400bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:29.910834 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 984/1950] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:29 2022] DELETE /identity/v3/projects/51399ff174e34f2d8937c9a7197d837f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:29.912457 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b28a3-b9e3-4285-8239-3c72e95400bf admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:29.915120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-019b28a3-b9e3-4285-8239-3c72e95400bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:30.005095 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96dcc46-47ef-43c3-b127-b5ebec3f8000 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:30.010902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cec833-a942-4132-b432-e163eb1b2f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:30.011842 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cec833-a942-4132-b432-e163eb1b2f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:30.012239 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cec833-a942-4132-b432-e163eb1b2f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:30.012530 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cec833-a942-4132-b432-e163eb1b2f77 admin admin] PATH_INFO: `/v3/projects/d9e5b9f0a1d84dc78ead517eb1533265` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:30.014115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cec833-a942-4132-b432-e163eb1b2f77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9e5b9f0a1d84dc78ead517eb1533265)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:30.016141 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cec833-a942-4132-b432-e163eb1b2f77 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:30.027179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b28a3-b9e3-4285-8239-3c72e95400bf admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:30.032591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aeeeee-addc-4b40-882c-de4602d303fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:30.033338 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aeeeee-addc-4b40-882c-de4602d303fa admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:30.033779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aeeeee-addc-4b40-882c-de4602d303fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:30.034064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aeeeee-addc-4b40-882c-de4602d303fa admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:30.034776 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aeeeee-addc-4b40-882c-de4602d303fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:30.036798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aeeeee-addc-4b40-882c-de4602d303fa admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:30.072190 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 985/1951] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:29 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:30.075525 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321318cc-85db-4cf3-8c14-15a27e45d8fd None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:30.076123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321318cc-85db-4cf3-8c14-15a27e45d8fd None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:30.076472 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321318cc-85db-4cf3-8c14-15a27e45d8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:30.086562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-35cec833-a942-4132-b432-e163eb1b2f77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9e5b9f0a1d84dc78ead517eb1533265', 'request_id': 'req-35cec833-a942-4132-b432-e163eb1b2f77'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:30.096702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-35cec833-a942-4132-b432-e163eb1b2f77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9e5b9f0a1d84dc78ead517eb1533265', 'request_id': 'req-35cec833-a942-4132-b432-e163eb1b2f77'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:30.111209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-35cec833-a942-4132-b432-e163eb1b2f77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9e5b9f0a1d84dc78ead517eb1533265', 'request_id': 'req-35cec833-a942-4132-b432-e163eb1b2f77'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:30.112741 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-321318cc-85db-4cf3-8c14-15a27e45d8fd None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:30.114219 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-321318cc-85db-4cf3-8c14-15a27e45d8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:30.134743 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-35cec833-a942-4132-b432-e163eb1b2f77 admin admin] The token cache is being invalidate because project d9e5b9f0a1d84dc78ead517eb1533265 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:30.135169 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-35cec833-a942-4132-b432-e163eb1b2f77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:30.138182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-35cec833-a942-4132-b432-e163eb1b2f77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:30.140277 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 967/1952] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:29 2022] DELETE /identity/v3/projects/d9e5b9f0a1d84dc78ead517eb1533265 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:30.141225 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cec833-a942-4132-b432-e163eb1b2f77 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:30.152659 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-35cec833-a942-4132-b432-e163eb1b2f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:30.154517 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:30.230235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 986/1953] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:30.237853 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321318cc-85db-4cf3-8c14-15a27e45d8fd None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:30.245930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-321318cc-85db-4cf3-8c14-15a27e45d8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:30.259321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321318cc-85db-4cf3-8c14-15a27e45d8fd None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:30.267389 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef4f40-8d2e-48be-ad7d-7e2c1d230243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:30.268928 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef4f40-8d2e-48be-ad7d-7e2c1d230243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:30.269483 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef4f40-8d2e-48be-ad7d-7e2c1d230243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:30.270003 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef4f40-8d2e-48be-ad7d-7e2c1d230243 admin admin] PATH_INFO: `/v3/users/750c131578cb415ea362cdef59d09407` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:30.271924 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef4f40-8d2e-48be-ad7d-7e2c1d230243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=750c131578cb415ea362cdef59d09407)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:30.276755 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef4f40-8d2e-48be-ad7d-7e2c1d230243 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:30.306179 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cec833-a942-4132-b432-e163eb1b2f77 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:30.312562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd6e77-39bb-4052-ba36-36a56d7c22d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:30.313461 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd6e77-39bb-4052-ba36-36a56d7c22d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:30.313749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd6e77-39bb-4052-ba36-36a56d7c22d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:30.314025 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd6e77-39bb-4052-ba36-36a56d7c22d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:30.314768 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd6e77-39bb-4052-ba36-36a56d7c22d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:30.317691 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd6e77-39bb-4052-ba36-36a56d7c22d0 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:30.336050 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 968/1954] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:30.340615 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9929fd-e77c-42de-ab96-16253d108609 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:30.341151 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9929fd-e77c-42de-ab96-16253d108609 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:30.341295 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9929fd-e77c-42de-ab96-16253d108609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:30.364303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-cbef4f40-8d2e-48be-ad7d-7e2c1d230243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '750c131578cb415ea362cdef59d09407'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:30.384946 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-1d9929fd-e77c-42de-ab96-16253d108609 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:30.386185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1d9929fd-e77c-42de-ab96-16253d108609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:30.388041 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-cbef4f40-8d2e-48be-ad7d-7e2c1d230243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '750c131578cb415ea362cdef59d09407'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:30.397272 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-cbef4f40-8d2e-48be-ad7d-7e2c1d230243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '750c131578cb415ea362cdef59d09407'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:30.401732 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 969/1955] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:30.402996 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9929fd-e77c-42de-ab96-16253d108609 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:30.409456 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-cbef4f40-8d2e-48be-ad7d-7e2c1d230243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '750c131578cb415ea362cdef59d09407'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:30.411871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-cbef4f40-8d2e-48be-ad7d-7e2c1d230243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '750c131578cb415ea362cdef59d09407'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:30.418211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9929fd-e77c-42de-ab96-16253d108609 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:30.421130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 987/1956] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:30 2022] DELETE /identity/v3/users/750c131578cb415ea362cdef59d09407 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:30.422125 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef4f40-8d2e-48be-ad7d-7e2c1d230243 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:30.423758 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-cbef4f40-8d2e-48be-ad7d-7e2c1d230243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:30.447277 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:30.528908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42b254-b03a-4495-93a1-61f057b2441a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:30.531269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42b254-b03a-4495-93a1-61f057b2441a admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:30.531594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42b254-b03a-4495-93a1-61f057b2441a admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:30.531937 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42b254-b03a-4495-93a1-61f057b2441a admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:30.532728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42b254-b03a-4495-93a1-61f057b2441a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:30.536620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42b254-b03a-4495-93a1-61f057b2441a admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:30.552893 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 970/1957] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:30.555940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42b254-b03a-4495-93a1-61f057b2441a admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:30.558591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-9c42b254-b03a-4495-93a1-61f057b2441a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:30.571014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef4f40-8d2e-48be-ad7d-7e2c1d230243 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:30.583280 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010722d-131e-43b9-97e6-2067ee76fb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:30.584275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010722d-131e-43b9-97e6-2067ee76fb3d admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:30.584275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010722d-131e-43b9-97e6-2067ee76fb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:30.584798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010722d-131e-43b9-97e6-2067ee76fb3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:30.585271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010722d-131e-43b9-97e6-2067ee76fb3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:30.585652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010722d-131e-43b9-97e6-2067ee76fb3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:30.587451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010722d-131e-43b9-97e6-2067ee76fb3d admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:30.599472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42b254-b03a-4495-93a1-61f057b2441a admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:30.610214 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 988/1958] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:06:30.614134 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386f56f-76bf-4af2-97ba-b1c1a2a1c6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:30.615308 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010722d-131e-43b9-97e6-2067ee76fb3d admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:30.620909 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386f56f-76bf-4af2-97ba-b1c1a2a1c6f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:30.622368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386f56f-76bf-4af2-97ba-b1c1a2a1c6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:30.623220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386f56f-76bf-4af2-97ba-b1c1a2a1c6f9 admin admin] PATH_INFO: `/v3/projects/71cbf4330a98405da75453640e6f2a41/users/3977f504291a4143a235991b6dc71282/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:30.635127 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010722d-131e-43b9-97e6-2067ee76fb3d admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:30.641768 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69db47a-5c12-43ec-89f2-d8a68c857381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:30.642556 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69db47a-5c12-43ec-89f2-d8a68c857381 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:30.642908 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69db47a-5c12-43ec-89f2-d8a68c857381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:30.643243 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69db47a-5c12-43ec-89f2-d8a68c857381 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:30.644007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69db47a-5c12-43ec-89f2-d8a68c857381 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:30.646426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69db47a-5c12-43ec-89f2-d8a68c857381 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:30.666487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386f56f-76bf-4af2-97ba-b1c1a2a1c6f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71cbf4330a98405da75453640e6f2a41, user_id=3977f504291a4143a235991b6dc71282, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:30.670212 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386f56f-76bf-4af2-97ba-b1c1a2a1c6f9 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:30.688411 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 971/1959] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:30 2022] PUT /identity/v3/projects/71cbf4330a98405da75453640e6f2a41/users/3977f504291a4143a235991b6dc71282/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:30.698236 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 989/1960] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:30 2022] POST /identity/v3/projects => generated 381 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:30.702398 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386f56f-76bf-4af2-97ba-b1c1a2a1c6f9 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:30.704694 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e454a-e854-4bbb-b955-203b898b771d None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:30.705133 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e454a-e854-4bbb-b955-203b898b771d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:30.705418 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e454a-e854-4bbb-b955-203b898b771d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:30.734317 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:30.737731 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-464e454a-e854-4bbb-b955-203b898b771d None None] MFA Rules not processed for user `3977f504291a4143a235991b6dc71282`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:30.739746 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-464e454a-e854-4bbb-b955-203b898b771d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:30.795768 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386f56f-76bf-4af2-97ba-b1c1a2a1c6f9 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:30.837614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 990/1961] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:30.842067 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e454a-e854-4bbb-b955-203b898b771d None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:30.863373 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520aace0-6d7a-41f4-85ab-5dd78d0eb39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:30.864320 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520aace0-6d7a-41f4-85ab-5dd78d0eb39c admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:30.864817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520aace0-6d7a-41f4-85ab-5dd78d0eb39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:30.865234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520aace0-6d7a-41f4-85ab-5dd78d0eb39c admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:30.866143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520aace0-6d7a-41f4-85ab-5dd78d0eb39c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:30.868808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520aace0-6d7a-41f4-85ab-5dd78d0eb39c admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:30.895806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 972/1962] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:30 2022] POST /identity/v3/users => generated 291 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:30.900740 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520aace0-6d7a-41f4-85ab-5dd78d0eb39c admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:30.913172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520aace0-6d7a-41f4-85ab-5dd78d0eb39c admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:30.921095 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7e4f2-f7e8-4918-a9c5-2ebcf59fba18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:30.921896 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7e4f2-f7e8-4918-a9c5-2ebcf59fba18 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:30.922235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7e4f2-f7e8-4918-a9c5-2ebcf59fba18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:30.922561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7e4f2-f7e8-4918-a9c5-2ebcf59fba18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:30.923527 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd7e4f2-f7e8-4918-a9c5-2ebcf59fba18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:30.925244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd7e4f2-f7e8-4918-a9c5-2ebcf59fba18 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:30.937409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 973/1963] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:30.941173 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7e4f2-f7e8-4918-a9c5-2ebcf59fba18 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:30.947930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e454a-e854-4bbb-b955-203b898b771d None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:30.952757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ad3c2-bb1b-498c-b16f-8e9e29ad381a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:30.953431 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ad3c2-bb1b-498c-b16f-8e9e29ad381a admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:30.954085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ad3c2-bb1b-498c-b16f-8e9e29ad381a admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:30.954085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ad3c2-bb1b-498c-b16f-8e9e29ad381a admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:30.954675 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ad3c2-bb1b-498c-b16f-8e9e29ad381a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:30.956188 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7e4f2-f7e8-4918-a9c5-2ebcf59fba18 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:30.956577 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ad3c2-bb1b-498c-b16f-8e9e29ad381a admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:30.962195 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7a2ae-966e-4c27-8c2a-801002c07c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:30.963170 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7a2ae-966e-4c27-8c2a-801002c07c8c admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:30.963691 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7a2ae-966e-4c27-8c2a-801002c07c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:30.964177 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7a2ae-966e-4c27-8c2a-801002c07c8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:30.965158 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7a2ae-966e-4c27-8c2a-801002c07c8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:30.968227 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7a2ae-966e-4c27-8c2a-801002c07c8c admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:30.974885 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:30.984058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 974/1964] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:06:30.987649 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7a2ae-966e-4c27-8c2a-801002c07c8c admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:30.993118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 991/1965] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:30 2022] POST /identity/v3/projects => generated 379 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:30.997238 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ad3c2-bb1b-498c-b16f-8e9e29ad381a admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:31.000864 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7a2ae-966e-4c27-8c2a-801002c07c8c admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:31.066275 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061f2f5-e5bd-430e-9f97-09263cf10aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:31.067058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061f2f5-e5bd-430e-9f97-09263cf10aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:31.067531 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061f2f5-e5bd-430e-9f97-09263cf10aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:31.067637 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061f2f5-e5bd-430e-9f97-09263cf10aa0 admin admin] PATH_INFO: `/v3/projects/2e0144505d2741908bc0098e6410ab4d/users/72be826af04a4bd9aabe890eef62880e/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:31.078724 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ad3c2-bb1b-498c-b16f-8e9e29ad381a admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:31.084289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061f2f5-e5bd-430e-9f97-09263cf10aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e0144505d2741908bc0098e6410ab4d, user_id=72be826af04a4bd9aabe890eef62880e, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:31.085805 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e0cd6-76cf-4623-b802-93207f0dafb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:31.087058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e0cd6-76cf-4623-b802-93207f0dafb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:31.087383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061f2f5-e5bd-430e-9f97-09263cf10aa0 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:31.087788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e0cd6-76cf-4623-b802-93207f0dafb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:31.088271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e0cd6-76cf-4623-b802-93207f0dafb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:31.089204 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e0cd6-76cf-4623-b802-93207f0dafb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:31.091978 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e0cd6-76cf-4623-b802-93207f0dafb6 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:31.102201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 975/1966] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:30 2022] PUT /identity/v3/projects/2e0144505d2741908bc0098e6410ab4d/users/72be826af04a4bd9aabe890eef62880e/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:31.103651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061f2f5-e5bd-430e-9f97-09263cf10aa0 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:31.113882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1061f2f5-e5bd-430e-9f97-09263cf10aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:31.124439 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 992/1967] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:30 2022] POST /identity/v3/users => generated 296 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:06:31.126103 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ead0f-71ed-4a0c-886f-2d0b7eebe357 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:31.126392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ead0f-71ed-4a0c-886f-2d0b7eebe357 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:31.126672 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ead0f-71ed-4a0c-886f-2d0b7eebe357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:31.156934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-fc0ead0f-71ed-4a0c-886f-2d0b7eebe357 None None] MFA Rules not processed for user `72be826af04a4bd9aabe890eef62880e`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:31.158203 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-fc0ead0f-71ed-4a0c-886f-2d0b7eebe357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:31.164460 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:31.199532 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061f2f5-e5bd-430e-9f97-09263cf10aa0 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:31.204229 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624daa5-9a54-462b-a0db-852a86a8aa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:31.205145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624daa5-9a54-462b-a0db-852a86a8aa7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:31.205427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624daa5-9a54-462b-a0db-852a86a8aa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:31.205698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624daa5-9a54-462b-a0db-852a86a8aa7b admin admin] PATH_INFO: `/v3/projects/f5fd162694e6435bb003f05fc46390d4` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:31.206902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624daa5-9a54-462b-a0db-852a86a8aa7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5fd162694e6435bb003f05fc46390d4)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:31.208830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624daa5-9a54-462b-a0db-852a86a8aa7b admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:31.237708 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 993/1968] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:31.238832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ead0f-71ed-4a0c-886f-2d0b7eebe357 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:31.258828 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ead0f-71ed-4a0c-886f-2d0b7eebe357 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:31.275259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0624daa5-9a54-462b-a0db-852a86a8aa7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5fd162694e6435bb003f05fc46390d4', 'request_id': 'req-0624daa5-9a54-462b-a0db-852a86a8aa7b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:31.283406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0624daa5-9a54-462b-a0db-852a86a8aa7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5fd162694e6435bb003f05fc46390d4', 'request_id': 'req-0624daa5-9a54-462b-a0db-852a86a8aa7b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:31.296465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0624daa5-9a54-462b-a0db-852a86a8aa7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5fd162694e6435bb003f05fc46390d4', 'request_id': 'req-0624daa5-9a54-462b-a0db-852a86a8aa7b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:31.313047 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0624daa5-9a54-462b-a0db-852a86a8aa7b admin admin] The token cache is being invalidate because project f5fd162694e6435bb003f05fc46390d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:31.313280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0624daa5-9a54-462b-a0db-852a86a8aa7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:31.314302 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0624daa5-9a54-462b-a0db-852a86a8aa7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:31.316912 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 976/1969] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:31 2022] DELETE /identity/v3/projects/f5fd162694e6435bb003f05fc46390d4 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:31.324253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624daa5-9a54-462b-a0db-852a86a8aa7b admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:31.325926 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0624daa5-9a54-462b-a0db-852a86a8aa7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:31.336465 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a3433-00da-4deb-9d0b-c55ed1f16138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:31.337768 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a3433-00da-4deb-9d0b-c55ed1f16138 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:31.338368 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a3433-00da-4deb-9d0b-c55ed1f16138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:31.338835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a3433-00da-4deb-9d0b-c55ed1f16138 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:31.339730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a3433-00da-4deb-9d0b-c55ed1f16138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:31.342046 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a3433-00da-4deb-9d0b-c55ed1f16138 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:31.354534 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:31.355624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 994/1970] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:31.359854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b94cd2-485b-4d83-8ec5-7e210f512a63 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:31.359854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b94cd2-485b-4d83-8ec5-7e210f512a63 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:31.359854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b94cd2-485b-4d83-8ec5-7e210f512a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:31.383131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-27b94cd2-485b-4d83-8ec5-7e210f512a63 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:31.385272 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-27b94cd2-485b-4d83-8ec5-7e210f512a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:31.409429 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 995/1971] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:31.409429 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b94cd2-485b-4d83-8ec5-7e210f512a63 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:31.409429 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-27b94cd2-485b-4d83-8ec5-7e210f512a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:31.443785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b94cd2-485b-4d83-8ec5-7e210f512a63 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:31.457141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71b4d2-07e8-4e6f-8b39-42815ee6d3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:31.461635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71b4d2-07e8-4e6f-8b39-42815ee6d3b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:31.461929 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71b4d2-07e8-4e6f-8b39-42815ee6d3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:31.462222 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71b4d2-07e8-4e6f-8b39-42815ee6d3b1 admin admin] PATH_INFO: `/v3/projects/e6b1c88d982b47918df97b0392f18a74/users/99137ccf6dc845c696c276c5f1d9e4cf/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:31.467152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624daa5-9a54-462b-a0db-852a86a8aa7b admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:31.474176 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a74b1-8dc9-4f28-8329-dfe4ab485b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:31.475175 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a74b1-8dc9-4f28-8329-dfe4ab485b99 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:31.475678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a74b1-8dc9-4f28-8329-dfe4ab485b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:31.477043 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a74b1-8dc9-4f28-8329-dfe4ab485b99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:31.477960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a74b1-8dc9-4f28-8329-dfe4ab485b99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:31.480933 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a74b1-8dc9-4f28-8329-dfe4ab485b99 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:31.517280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 977/1972] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:31 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:31.518955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75053acb-a36c-4941-b1cc-50aa105c62a0 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:31.519297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75053acb-a36c-4941-b1cc-50aa105c62a0 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:31.519577 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75053acb-a36c-4941-b1cc-50aa105c62a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:31.545424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71b4d2-07e8-4e6f-8b39-42815ee6d3b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6b1c88d982b47918df97b0392f18a74, user_id=99137ccf6dc845c696c276c5f1d9e4cf, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:31.549719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71b4d2-07e8-4e6f-8b39-42815ee6d3b1 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:31.551132 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-75053acb-a36c-4941-b1cc-50aa105c62a0 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:31.552630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-75053acb-a36c-4941-b1cc-50aa105c62a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:31.586849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 996/1973] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:31 2022] PUT /identity/v3/projects/e6b1c88d982b47918df97b0392f18a74/users/99137ccf6dc845c696c276c5f1d9e4cf/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:31.589453 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71b4d2-07e8-4e6f-8b39-42815ee6d3b1 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:31.643853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 978/1974] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:31.643853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75053acb-a36c-4941-b1cc-50aa105c62a0 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:31.652402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75053acb-a36c-4941-b1cc-50aa105c62a0 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:31.671484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71b4d2-07e8-4e6f-8b39-42815ee6d3b1 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:31.677365 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbdddf-6de9-4cba-aaa1-7cb9b1cb4210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:31.678151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbdddf-6de9-4cba-aaa1-7cb9b1cb4210 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:31.678510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbdddf-6de9-4cba-aaa1-7cb9b1cb4210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:31.678865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbdddf-6de9-4cba-aaa1-7cb9b1cb4210 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:31.680389 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbdddf-6de9-4cba-aaa1-7cb9b1cb4210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:31.682890 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbdddf-6de9-4cba-aaa1-7cb9b1cb4210 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:31.709711 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 997/1975] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:31 2022] POST /identity/v3/users => generated 297 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:31.711339 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbdddf-6de9-4cba-aaa1-7cb9b1cb4210 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:31.718603 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:31.724628 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbdddf-6de9-4cba-aaa1-7cb9b1cb4210 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:31.734536 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc873a-c793-441d-9d85-17fcb980ded6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:31.746423 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc873a-c793-441d-9d85-17fcb980ded6 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:31.746728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc873a-c793-441d-9d85-17fcb980ded6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:31.747021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc873a-c793-441d-9d85-17fcb980ded6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:31.749711 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc873a-c793-441d-9d85-17fcb980ded6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:31.750042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc873a-c793-441d-9d85-17fcb980ded6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:31.753736 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc873a-c793-441d-9d85-17fcb980ded6 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:31.772542 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c7835-c028-4580-b7d3-75eff5f29a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:31.774190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c7835-c028-4580-b7d3-75eff5f29a3b admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:31.774685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c7835-c028-4580-b7d3-75eff5f29a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:31.775206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c7835-c028-4580-b7d3-75eff5f29a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:31.776357 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c7835-c028-4580-b7d3-75eff5f29a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:31.776614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 998/1976] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:06:31.777827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc873a-c793-441d-9d85-17fcb980ded6 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:31.780488 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c7835-c028-4580-b7d3-75eff5f29a3b admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:31.797341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 979/1977] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:31.798470 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c7835-c028-4580-b7d3-75eff5f29a3b admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:31.805599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc873a-c793-441d-9d85-17fcb980ded6 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:31.820363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bab335-8ca9-4d52-b3fc-e1d3e3f4fdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:31.822586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bab335-8ca9-4d52-b3fc-e1d3e3f4fdd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:31.823559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bab335-8ca9-4d52-b3fc-e1d3e3f4fdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:31.825426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bab335-8ca9-4d52-b3fc-e1d3e3f4fdd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:31.828098 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c7835-c028-4580-b7d3-75eff5f29a3b admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:31.828650 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bab335-8ca9-4d52-b3fc-e1d3e3f4fdd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:31.832462 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bab335-8ca9-4d52-b3fc-e1d3e3f4fdd6 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:31.850432 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575f4d1-530f-4f1e-8474-1bc87daecfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:31.851961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575f4d1-530f-4f1e-8474-1bc87daecfa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:31.852794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575f4d1-530f-4f1e-8474-1bc87daecfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:31.853576 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575f4d1-530f-4f1e-8474-1bc87daecfa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:31.854675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575f4d1-530f-4f1e-8474-1bc87daecfa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:31.858152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575f4d1-530f-4f1e-8474-1bc87daecfa7 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:31.866465 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 999/1978] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:31.867863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bab335-8ca9-4d52-b3fc-e1d3e3f4fdd6 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:31.890200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bab335-8ca9-4d52-b3fc-e1d3e3f4fdd6 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:31.917064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa82094-0bf6-45bc-ac21-7903219ffc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:31.918516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa82094-0bf6-45bc-ac21-7903219ffc02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:31.918516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa82094-0bf6-45bc-ac21-7903219ffc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:31.918816 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa82094-0bf6-45bc-ac21-7903219ffc02 admin admin] PATH_INFO: `/v3/projects/e6b1c88d982b47918df97b0392f18a74/users/99137ccf6dc845c696c276c5f1d9e4cf/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:31.928239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa82094-0bf6-45bc-ac21-7903219ffc02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6b1c88d982b47918df97b0392f18a74, user_id=99137ccf6dc845c696c276c5f1d9e4cf, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:31.929973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 980/1979] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:31 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:31.933005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575f4d1-530f-4f1e-8474-1bc87daecfa7 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:31.933964 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa82094-0bf6-45bc-ac21-7903219ffc02 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:31.972184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1000/1980] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:31 2022] PUT /identity/v3/projects/e6b1c88d982b47918df97b0392f18a74/users/99137ccf6dc845c696c276c5f1d9e4cf/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:31.972184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa82094-0bf6-45bc-ac21-7903219ffc02 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:31.974021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:32.105164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575f4d1-530f-4f1e-8474-1bc87daecfa7 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:32.164302 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa82094-0bf6-45bc-ac21-7903219ffc02 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:32.175955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63fcd0a-95ca-4ad3-aae6-45b5ae6f1e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:32.176833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fcd0a-95ca-4ad3-aae6-45b5ae6f1e76 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:32.177120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fcd0a-95ca-4ad3-aae6-45b5ae6f1e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:32.177398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fcd0a-95ca-4ad3-aae6-45b5ae6f1e76 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:32.178104 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63fcd0a-95ca-4ad3-aae6-45b5ae6f1e76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:32.181295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63fcd0a-95ca-4ad3-aae6-45b5ae6f1e76 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:32.219516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1001/1981] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:31 2022] POST /identity/v3/users => generated 286 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:32.220700 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b32222-0cc3-4954-a646-c1f88087571a None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:32.220700 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b32222-0cc3-4954-a646-c1f88087571a None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:32.222958 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b32222-0cc3-4954-a646-c1f88087571a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:32.237438 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dece10-f473-4828-8c22-43c7c865d338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:32.238359 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dece10-f473-4828-8c22-43c7c865d338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:32.238622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dece10-f473-4828-8c22-43c7c865d338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:32.238918 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dece10-f473-4828-8c22-43c7c865d338 admin admin] PATH_INFO: `/v3/projects/93486daa515540e7aa5a340f60898f89/users/b1258c0752ee4d099a2c2372204561be/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:32.256637 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dece10-f473-4828-8c22-43c7c865d338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93486daa515540e7aa5a340f60898f89, user_id=b1258c0752ee4d099a2c2372204561be, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:32.258584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-90b32222-0cc3-4954-a646-c1f88087571a None None] MFA Rules not processed for user `99137ccf6dc845c696c276c5f1d9e4cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:32.260106 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-90b32222-0cc3-4954-a646-c1f88087571a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:32.260216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dece10-f473-4828-8c22-43c7c865d338 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:32.276554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 981/1982] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:31 2022] PUT /identity/v3/projects/93486daa515540e7aa5a340f60898f89/users/b1258c0752ee4d099a2c2372204561be/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:32.277428 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dece10-f473-4828-8c22-43c7c865d338 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:32.357761 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1002/1983] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:32.361748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b32222-0cc3-4954-a646-c1f88087571a None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:32.373569 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dece10-f473-4828-8c22-43c7c865d338 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:32.379024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c0061-1dcf-4c96-a2f3-c71979181c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:32.379707 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c0061-1dcf-4c96-a2f3-c71979181c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:32.380000 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c0061-1dcf-4c96-a2f3-c71979181c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:32.380283 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c0061-1dcf-4c96-a2f3-c71979181c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:32.381300 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7c0061-1dcf-4c96-a2f3-c71979181c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:32.382699 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7c0061-1dcf-4c96-a2f3-c71979181c57 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:32.385767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b32222-0cc3-4954-a646-c1f88087571a None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:32.391929 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e961d1-e396-4b54-a79e-38d7516cab09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:32.392621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e961d1-e396-4b54-a79e-38d7516cab09 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:32.392959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e961d1-e396-4b54-a79e-38d7516cab09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:32.393492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e961d1-e396-4b54-a79e-38d7516cab09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:32.394243 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e961d1-e396-4b54-a79e-38d7516cab09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:32.395022 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 982/1984] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:32.396209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c0061-1dcf-4c96-a2f3-c71979181c57 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:32.398711 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6a7c0061-1dcf-4c96-a2f3-c71979181c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:32.399454 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e961d1-e396-4b54-a79e-38d7516cab09 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:32.415116 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1003/1985] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:32.416013 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e961d1-e396-4b54-a79e-38d7516cab09 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:32.428872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e961d1-e396-4b54-a79e-38d7516cab09 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:32.434838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a7c95-7581-4596-b2a5-6731f6423780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:32.435517 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a7c95-7581-4596-b2a5-6731f6423780 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:32.435803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a7c95-7581-4596-b2a5-6731f6423780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:32.436084 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a7c95-7581-4596-b2a5-6731f6423780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:32.436851 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a7c95-7581-4596-b2a5-6731f6423780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:32.438468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a7c95-7581-4596-b2a5-6731f6423780 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:32.450170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1004/1986] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:32.450811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a7c95-7581-4596-b2a5-6731f6423780 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:32.457645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:32.464602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a7c95-7581-4596-b2a5-6731f6423780 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:32.469242 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348dea7-8025-457a-966b-d3d925db6e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:32.470135 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348dea7-8025-457a-966b-d3d925db6e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:32.470419 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348dea7-8025-457a-966b-d3d925db6e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:32.470695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348dea7-8025-457a-966b-d3d925db6e69 admin admin] PATH_INFO: `/v3/projects/93486daa515540e7aa5a340f60898f89/users/b1258c0752ee4d099a2c2372204561be/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:32.472681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348dea7-8025-457a-966b-d3d925db6e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93486daa515540e7aa5a340f60898f89, user_id=b1258c0752ee4d099a2c2372204561be, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:32.475204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c0061-1dcf-4c96-a2f3-c71979181c57 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:32.475681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348dea7-8025-457a-966b-d3d925db6e69 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:32.479173 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4d67e-5d74-4fd4-a5ea-971a969bd5c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:32.479825 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4d67e-5d74-4fd4-a5ea-971a969bd5c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:32.480155 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4d67e-5d74-4fd4-a5ea-971a969bd5c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:32.480449 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4d67e-5d74-4fd4-a5ea-971a969bd5c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:32.488572 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1005/1987] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:32 2022] PUT /identity/v3/projects/93486daa515540e7aa5a340f60898f89/users/b1258c0752ee4d099a2c2372204561be/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:32.489512 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348dea7-8025-457a-966b-d3d925db6e69 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:32.496107 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:32.551302 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4d67e-5d74-4fd4-a5ea-971a969bd5c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:32.552810 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4d67e-5d74-4fd4-a5ea-971a969bd5c1 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:32.555893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348dea7-8025-457a-966b-d3d925db6e69 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:32.560625 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e222d2-bd7b-4da4-bdac-c6112ef52255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:32.561666 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e222d2-bd7b-4da4-bdac-c6112ef52255 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:32.561956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e222d2-bd7b-4da4-bdac-c6112ef52255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:32.562231 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e222d2-bd7b-4da4-bdac-c6112ef52255 admin admin] PATH_INFO: `/v3/projects/4ef84c2db15e477fa1434be7b5e02a8a/users/55f21f40ad5d4cb0801b0d2aaf02c8cc/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:32.563722 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 983/1988] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:32.566120 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3bdc6-3170-45e8-934d-d79d5d650573 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:32.566430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3bdc6-3170-45e8-934d-d79d5d650573 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:32.566776 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3bdc6-3170-45e8-934d-d79d5d650573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:32.577889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e222d2-bd7b-4da4-bdac-c6112ef52255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ef84c2db15e477fa1434be7b5e02a8a, user_id=55f21f40ad5d4cb0801b0d2aaf02c8cc, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:32.580501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e222d2-bd7b-4da4-bdac-c6112ef52255 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:32.596798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1006/1989] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:32 2022] PUT /identity/v3/projects/4ef84c2db15e477fa1434be7b5e02a8a/users/55f21f40ad5d4cb0801b0d2aaf02c8cc/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:32.604820 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-fac3bdc6-3170-45e8-934d-d79d5d650573 None None] MFA Rules not processed for user `b1258c0752ee4d099a2c2372204561be`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:32.609004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-fac3bdc6-3170-45e8-934d-d79d5d650573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:32.610444 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba78ae-84f0-4e36-b7d2-b79e9dfd708d None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:32.610741 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba78ae-84f0-4e36-b7d2-b79e9dfd708d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:32.611051 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba78ae-84f0-4e36-b7d2-b79e9dfd708d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:32.642681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-86ba78ae-84f0-4e36-b7d2-b79e9dfd708d None None] MFA Rules not processed for user `55f21f40ad5d4cb0801b0d2aaf02c8cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:32.645274 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-86ba78ae-84f0-4e36-b7d2-b79e9dfd708d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:32.702068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 984/1990] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:32.702889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3bdc6-3170-45e8-934d-d79d5d650573 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:32.741173 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1007/1991] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:32.781458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3bdc6-3170-45e8-934d-d79d5d650573 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:32.785347 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f35ab-6e49-43a2-a801-b9278535d18a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:32.786068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f35ab-6e49-43a2-a801-b9278535d18a service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:32.786357 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f35ab-6e49-43a2-a801-b9278535d18a service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:32.786662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f35ab-6e49-43a2-a801-b9278535d18a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:32.788804 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-249f35ab-6e49-43a2-a801-b9278535d18a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:32.824581 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:32.859811 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f35ab-6e49-43a2-a801-b9278535d18a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:32.861360 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f35ab-6e49-43a2-a801-b9278535d18a service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:32.871577 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 985/1992] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:32.901397 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba78ae-84f0-4e36-b7d2-b79e9dfd708d None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:32.911673 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba78ae-84f0-4e36-b7d2-b79e9dfd708d None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:32.915692 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439cb93-3d69-420a-b3ee-0e711c9d4792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:32.916415 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439cb93-3d69-420a-b3ee-0e711c9d4792 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:32.916611 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439cb93-3d69-420a-b3ee-0e711c9d4792 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:32.916949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439cb93-3d69-420a-b3ee-0e711c9d4792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:32.927114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439cb93-3d69-420a-b3ee-0e711c9d4792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:32.928627 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439cb93-3d69-420a-b3ee-0e711c9d4792 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:32.939133 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1008/1993] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:32.946919 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f35ab-6e49-43a2-a801-b9278535d18a service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:32.948473 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-249f35ab-6e49-43a2-a801-b9278535d18a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:32.960298 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f35ab-6e49-43a2-a801-b9278535d18a service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:32.965000 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1994ead4-3cec-4975-a424-e3ad45d945fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:32.965853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994ead4-3cec-4975-a424-e3ad45d945fc service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:32.965853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994ead4-3cec-4975-a424-e3ad45d945fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:32.966059 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994ead4-3cec-4975-a424-e3ad45d945fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:32.976249 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1994ead4-3cec-4975-a424-e3ad45d945fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:32.977729 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1994ead4-3cec-4975-a424-e3ad45d945fc service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:32.988135 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 986/1994] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:06:33.009655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439cb93-3d69-420a-b3ee-0e711c9d4792 service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:33.091978 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439cb93-3d69-420a-b3ee-0e711c9d4792 service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:33.098577 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cff474-4417-4578-8120-00d431827189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:33.100730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cff474-4417-4578-8120-00d431827189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:33.100730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cff474-4417-4578-8120-00d431827189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:33.100730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cff474-4417-4578-8120-00d431827189 admin admin] PATH_INFO: `/v3/users/55f21f40ad5d4cb0801b0d2aaf02c8cc` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:33.101965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cff474-4417-4578-8120-00d431827189 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55f21f40ad5d4cb0801b0d2aaf02c8cc)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:33.104099 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cff474-4417-4578-8120-00d431827189 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:33.110245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:33.177446 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-52cff474-4417-4578-8120-00d431827189 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55f21f40ad5d4cb0801b0d2aaf02c8cc'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:33.192986 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-52cff474-4417-4578-8120-00d431827189 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55f21f40ad5d4cb0801b0d2aaf02c8cc'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:33.203250 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-52cff474-4417-4578-8120-00d431827189 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55f21f40ad5d4cb0801b0d2aaf02c8cc'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:33.219077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-52cff474-4417-4578-8120-00d431827189 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55f21f40ad5d4cb0801b0d2aaf02c8cc'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:33.219805 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-52cff474-4417-4578-8120-00d431827189 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55f21f40ad5d4cb0801b0d2aaf02c8cc'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:33.244145 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1009/1995] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:33 2022] DELETE /identity/v3/users/55f21f40ad5d4cb0801b0d2aaf02c8cc => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:33.249029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787535a6-7798-4557-b8eb-589c15cdc25a None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:33.249326 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787535a6-7798-4557-b8eb-589c15cdc25a None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:33.249661 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787535a6-7798-4557-b8eb-589c15cdc25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:33.285648 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-787535a6-7798-4557-b8eb-589c15cdc25a None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:33.288199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-787535a6-7798-4557-b8eb-589c15cdc25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:33.417412 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 987/1996] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:33.427234 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cff474-4417-4578-8120-00d431827189 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:33.429147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-52cff474-4417-4578-8120-00d431827189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:33.584261 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cff474-4417-4578-8120-00d431827189 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:33.589682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f51187-759d-4155-9955-0d8d04847d83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:33.590381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f51187-759d-4155-9955-0d8d04847d83 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:33.590770 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f51187-759d-4155-9955-0d8d04847d83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:33.591175 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f51187-759d-4155-9955-0d8d04847d83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:33.608691 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f51187-759d-4155-9955-0d8d04847d83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:33.610568 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f51187-759d-4155-9955-0d8d04847d83 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:33.628031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1010/1997] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:33.877258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787535a6-7798-4557-b8eb-589c15cdc25a None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:33.879026 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-787535a6-7798-4557-b8eb-589c15cdc25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:33.901250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787535a6-7798-4557-b8eb-589c15cdc25a None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:33.914182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d6364-6a1a-4759-bf80-feae24560837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:33.916582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d6364-6a1a-4759-bf80-feae24560837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:33.917395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d6364-6a1a-4759-bf80-feae24560837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:33.917740 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d6364-6a1a-4759-bf80-feae24560837 admin admin] PATH_INFO: `/v3/users/72be826af04a4bd9aabe890eef62880e` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:33.921625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d6364-6a1a-4759-bf80-feae24560837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72be826af04a4bd9aabe890eef62880e)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:33.924852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d6364-6a1a-4759-bf80-feae24560837 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:34.026345 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2a7d6364-6a1a-4759-bf80-feae24560837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72be826af04a4bd9aabe890eef62880e'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:34.038705 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2a7d6364-6a1a-4759-bf80-feae24560837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72be826af04a4bd9aabe890eef62880e'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:34.043897 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f51187-759d-4155-9955-0d8d04847d83 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:34.045683 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-75f51187-759d-4155-9955-0d8d04847d83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:34.065016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f51187-759d-4155-9955-0d8d04847d83 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:34.067002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2a7d6364-6a1a-4759-bf80-feae24560837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72be826af04a4bd9aabe890eef62880e'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:34.067831 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2a7d6364-6a1a-4759-bf80-feae24560837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72be826af04a4bd9aabe890eef62880e'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:34.068460 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2a7d6364-6a1a-4759-bf80-feae24560837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72be826af04a4bd9aabe890eef62880e'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:34.079808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15ac94-c959-41eb-8f67-85c86abfe871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:34.080884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15ac94-c959-41eb-8f67-85c86abfe871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:34.081063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15ac94-c959-41eb-8f67-85c86abfe871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:34.081313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15ac94-c959-41eb-8f67-85c86abfe871 admin admin] PATH_INFO: `/v3/projects/4ef84c2db15e477fa1434be7b5e02a8a` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:34.083800 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15ac94-c959-41eb-8f67-85c86abfe871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ef84c2db15e477fa1434be7b5e02a8a)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:34.114544 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15ac94-c959-41eb-8f67-85c86abfe871 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:34.135980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 988/1998] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:33 2022] DELETE /identity/v3/users/72be826af04a4bd9aabe890eef62880e => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:34.140376 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843c4bf-b8b9-4271-a210-cfdcc9d8cade None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:34.140893 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843c4bf-b8b9-4271-a210-cfdcc9d8cade None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:34.141182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843c4bf-b8b9-4271-a210-cfdcc9d8cade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:34.166798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-4843c4bf-b8b9-4271-a210-cfdcc9d8cade None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:34.168322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4843c4bf-b8b9-4271-a210-cfdcc9d8cade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:34.200605 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-8a15ac94-c959-41eb-8f67-85c86abfe871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ef84c2db15e477fa1434be7b5e02a8a', 'request_id': 'req-8a15ac94-c959-41eb-8f67-85c86abfe871'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:34.206494 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:34.209092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-8a15ac94-c959-41eb-8f67-85c86abfe871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ef84c2db15e477fa1434be7b5e02a8a', 'request_id': 'req-8a15ac94-c959-41eb-8f67-85c86abfe871'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:34.223280 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-8a15ac94-c959-41eb-8f67-85c86abfe871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ef84c2db15e477fa1434be7b5e02a8a', 'request_id': 'req-8a15ac94-c959-41eb-8f67-85c86abfe871'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:34.236516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-8a15ac94-c959-41eb-8f67-85c86abfe871 admin admin] The token cache is being invalidate because project 4ef84c2db15e477fa1434be7b5e02a8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:34.236773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-8a15ac94-c959-41eb-8f67-85c86abfe871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:34.237459 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-8a15ac94-c959-41eb-8f67-85c86abfe871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:34.239228 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1011/1999] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:34 2022] DELETE /identity/v3/projects/4ef84c2db15e477fa1434be7b5e02a8a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:34.244424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2abc9-2e75-4fa0-8e33-3ab353c66c5b None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:34.244750 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2abc9-2e75-4fa0-8e33-3ab353c66c5b None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:34.245119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2abc9-2e75-4fa0-8e33-3ab353c66c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:34.259985 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 989/2000] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:34.266272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843c4bf-b8b9-4271-a210-cfdcc9d8cade None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:34.267692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4843c4bf-b8b9-4271-a210-cfdcc9d8cade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:34.286210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-21a2abc9-2e75-4fa0-8e33-3ab353c66c5b None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:34.288704 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-21a2abc9-2e75-4fa0-8e33-3ab353c66c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:34.295249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:34.334042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843c4bf-b8b9-4271-a210-cfdcc9d8cade None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:34.390483 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1012/2001] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:34.396472 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a0225-1c83-490d-9d08-211d57e6f56b None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:34.396803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a0225-1c83-490d-9d08-211d57e6f56b None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:34.397090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a0225-1c83-490d-9d08-211d57e6f56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:34.419183 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df72ad-ac89-456c-9131-973000bb8854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:34.419575 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df72ad-ac89-456c-9131-973000bb8854 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:34.419882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df72ad-ac89-456c-9131-973000bb8854 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:34.420166 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df72ad-ac89-456c-9131-973000bb8854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:34.431383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70df72ad-ac89-456c-9131-973000bb8854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:34.433056 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70df72ad-ac89-456c-9131-973000bb8854 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:34.446923 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 990/2002] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:06:34.447513 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-b61a0225-1c83-490d-9d08-211d57e6f56b None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:34.454441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b61a0225-1c83-490d-9d08-211d57e6f56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:34.474866 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1013/2003] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:34.475755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df72ad-ac89-456c-9131-973000bb8854 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:34.486991 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df72ad-ac89-456c-9131-973000bb8854 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:34.492343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310aea7d-7709-4af0-9265-dcd338afbc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:34.493231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310aea7d-7709-4af0-9265-dcd338afbc27 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:34.493539 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310aea7d-7709-4af0-9265-dcd338afbc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:34.493942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310aea7d-7709-4af0-9265-dcd338afbc27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:34.494687 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310aea7d-7709-4af0-9265-dcd338afbc27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:34.495001 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310aea7d-7709-4af0-9265-dcd338afbc27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:34.496928 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310aea7d-7709-4af0-9265-dcd338afbc27 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:34.511548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 991/2004] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 22:06:34.516581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a0225-1c83-490d-9d08-211d57e6f56b None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:34.529087 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a0225-1c83-490d-9d08-211d57e6f56b None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:34.535952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778f9ca-ae89-44e5-8e44-b4c6c01005a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:34.536749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778f9ca-ae89-44e5-8e44-b4c6c01005a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:34.536923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778f9ca-ae89-44e5-8e44-b4c6c01005a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:34.537202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778f9ca-ae89-44e5-8e44-b4c6c01005a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:34.537889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778f9ca-ae89-44e5-8e44-b4c6c01005a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:34.540339 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778f9ca-ae89-44e5-8e44-b4c6c01005a6 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:34.592419 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1014/2005] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:34 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:34.598460 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310aea7d-7709-4af0-9265-dcd338afbc27 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:34.614868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310aea7d-7709-4af0-9265-dcd338afbc27 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:34.777537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7974d-78cf-4eec-8ac4-7be66b5d5877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:34.779077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7974d-78cf-4eec-8ac4-7be66b5d5877 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:34.779648 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7974d-78cf-4eec-8ac4-7be66b5d5877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:34.780197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7974d-78cf-4eec-8ac4-7be66b5d5877 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:34.781696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e7974d-78cf-4eec-8ac4-7be66b5d5877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:34.785374 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e7974d-78cf-4eec-8ac4-7be66b5d5877 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:34.822604 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 992/2006] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:34 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:34.826515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778f9ca-ae89-44e5-8e44-b4c6c01005a6 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:34.863247 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778f9ca-ae89-44e5-8e44-b4c6c01005a6 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:34.870140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1678ae-b3eb-4abb-ba75-7eb92396b7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:34.870594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1678ae-b3eb-4abb-ba75-7eb92396b7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:34.870930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1678ae-b3eb-4abb-ba75-7eb92396b7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:34.871427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1678ae-b3eb-4abb-ba75-7eb92396b7a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:34.872373 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1678ae-b3eb-4abb-ba75-7eb92396b7a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:34.874122 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1678ae-b3eb-4abb-ba75-7eb92396b7a4 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:34.896586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7974d-78cf-4eec-8ac4-7be66b5d5877 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:34.897229 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1015/2007] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:34.904896 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:34.912005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7974d-78cf-4eec-8ac4-7be66b5d5877 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:34.938950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ab9c8-65a8-4427-9b8d-834a5b2875a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:34.939636 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ab9c8-65a8-4427-9b8d-834a5b2875a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:34.940105 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ab9c8-65a8-4427-9b8d-834a5b2875a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:34.940371 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ab9c8-65a8-4427-9b8d-834a5b2875a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:34.941180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ab9c8-65a8-4427-9b8d-834a5b2875a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:34.943383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ab9c8-65a8-4427-9b8d-834a5b2875a5 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:34.955477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 993/2008] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:34.962340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1678ae-b3eb-4abb-ba75-7eb92396b7a4 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:34.975616 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ab9c8-65a8-4427-9b8d-834a5b2875a5 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:34.977045 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1678ae-b3eb-4abb-ba75-7eb92396b7a4 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:34.978661 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-127ab9c8-65a8-4427-9b8d-834a5b2875a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:34.982778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae1daa-380a-48c9-85b5-23fa60382837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:34.983898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae1daa-380a-48c9-85b5-23fa60382837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:34.984241 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae1daa-380a-48c9-85b5-23fa60382837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:34.984605 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae1daa-380a-48c9-85b5-23fa60382837 admin admin] PATH_INFO: `/v3/projects/cb7e4dce11704c83a5dc447245fb8118/users/77a7cc2556ee46aca86ee8e8a86def0f/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:34.998040 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ab9c8-65a8-4427-9b8d-834a5b2875a5 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:35.009929 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:35.012032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:35.013400 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:35.013747 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80 admin admin] PATH_INFO: `/v3/projects/2e0144505d2741908bc0098e6410ab4d` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:35.015247 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae1daa-380a-48c9-85b5-23fa60382837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb7e4dce11704c83a5dc447245fb8118, user_id=77a7cc2556ee46aca86ee8e8a86def0f, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:35.015978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e0144505d2741908bc0098e6410ab4d)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:35.017920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:35.018414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae1daa-380a-48c9-85b5-23fa60382837 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:35.053109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1016/2009] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:34 2022] PUT /identity/v3/projects/cb7e4dce11704c83a5dc447245fb8118/users/77a7cc2556ee46aca86ee8e8a86def0f/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:35.058342 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a8d0e-7dab-43dd-b86e-b1a7bbf327a3 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:35.058626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a8d0e-7dab-43dd-b86e-b1a7bbf327a3 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:35.058937 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a8d0e-7dab-43dd-b86e-b1a7bbf327a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:35.086756 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e0144505d2741908bc0098e6410ab4d', 'request_id': 'req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:35.098845 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e0144505d2741908bc0098e6410ab4d', 'request_id': 'req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:35.114404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-846a8d0e-7dab-43dd-b86e-b1a7bbf327a3 None None] MFA Rules not processed for user `77a7cc2556ee46aca86ee8e8a86def0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:35.115996 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-846a8d0e-7dab-43dd-b86e-b1a7bbf327a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:35.120584 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e0144505d2741908bc0098e6410ab4d', 'request_id': 'req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:35.145379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80 admin admin] The token cache is being invalidate because project 2e0144505d2741908bc0098e6410ab4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:35.146716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:35.147632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:35.154459 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 994/2010] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:34 2022] DELETE /identity/v3/projects/2e0144505d2741908bc0098e6410ab4d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:35.156586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:35.163202 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:35.260144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1017/2011] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:35.261237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88648d-35a5-4e73-9c9f-5e12e9a61e80 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:35.268465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b02cb6-3d65-4c9d-9ccb-d7ae77554207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:35.269444 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b02cb6-3d65-4c9d-9ccb-d7ae77554207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:35.270177 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b02cb6-3d65-4c9d-9ccb-d7ae77554207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:35.270177 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b02cb6-3d65-4c9d-9ccb-d7ae77554207 admin admin] PATH_INFO: `/v3/users/b1258c0752ee4d099a2c2372204561be` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:35.271415 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b02cb6-3d65-4c9d-9ccb-d7ae77554207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1258c0752ee4d099a2c2372204561be)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:35.283445 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b02cb6-3d65-4c9d-9ccb-d7ae77554207 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:35.329547 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a8d0e-7dab-43dd-b86e-b1a7bbf327a3 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:35.332594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-846a8d0e-7dab-43dd-b86e-b1a7bbf327a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:35.358105 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a9b02cb6-3d65-4c9d-9ccb-d7ae77554207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1258c0752ee4d099a2c2372204561be'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:35.372068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a9b02cb6-3d65-4c9d-9ccb-d7ae77554207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1258c0752ee4d099a2c2372204561be'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:35.384390 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a9b02cb6-3d65-4c9d-9ccb-d7ae77554207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1258c0752ee4d099a2c2372204561be'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:35.385634 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a9b02cb6-3d65-4c9d-9ccb-d7ae77554207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1258c0752ee4d099a2c2372204561be'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:35.386917 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a9b02cb6-3d65-4c9d-9ccb-d7ae77554207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1258c0752ee4d099a2c2372204561be'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:35.404690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:35.408043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:35.416986 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 995/2012] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:35 2022] DELETE /identity/v3/users/b1258c0752ee4d099a2c2372204561be => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:35.442673 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a8d0e-7dab-43dd-b86e-b1a7bbf327a3 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:35.537348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b3224-cb63-49db-b6d4-e7e282facf4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:35.538357 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b3224-cb63-49db-b6d4-e7e282facf4f service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:35.538917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b3224-cb63-49db-b6d4-e7e282facf4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:35.539403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b3224-cb63-49db-b6d4-e7e282facf4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:35.541633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-199b3224-cb63-49db-b6d4-e7e282facf4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:35.631285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b3224-cb63-49db-b6d4-e7e282facf4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:35.634718 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b3224-cb63-49db-b6d4-e7e282facf4f service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:35.649978 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1018/2013] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:35.825089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b02cb6-3d65-4c9d-9ccb-d7ae77554207 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:35.827135 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a9b02cb6-3d65-4c9d-9ccb-d7ae77554207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:35.952955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b02cb6-3d65-4c9d-9ccb-d7ae77554207 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:35.962852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bf072-6285-46a8-b7e2-7cece329e7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:35.964075 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bf072-6285-46a8-b7e2-7cece329e7dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:35.964590 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bf072-6285-46a8-b7e2-7cece329e7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:35.965115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bf072-6285-46a8-b7e2-7cece329e7dd admin admin] PATH_INFO: `/v3/projects/93486daa515540e7aa5a340f60898f89` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:35.966912 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bf072-6285-46a8-b7e2-7cece329e7dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=93486daa515540e7aa5a340f60898f89)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:35.969519 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bf072-6285-46a8-b7e2-7cece329e7dd admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:36.036765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-d29bf072-6285-46a8-b7e2-7cece329e7dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93486daa515540e7aa5a340f60898f89', 'request_id': 'req-d29bf072-6285-46a8-b7e2-7cece329e7dd'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:36.049857 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-d29bf072-6285-46a8-b7e2-7cece329e7dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93486daa515540e7aa5a340f60898f89', 'request_id': 'req-d29bf072-6285-46a8-b7e2-7cece329e7dd'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:36.064529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-d29bf072-6285-46a8-b7e2-7cece329e7dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93486daa515540e7aa5a340f60898f89', 'request_id': 'req-d29bf072-6285-46a8-b7e2-7cece329e7dd'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:36.088460 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-d29bf072-6285-46a8-b7e2-7cece329e7dd admin admin] The token cache is being invalidate because project 93486daa515540e7aa5a340f60898f89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:36.089893 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-d29bf072-6285-46a8-b7e2-7cece329e7dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:36.091554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-d29bf072-6285-46a8-b7e2-7cece329e7dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:36.096424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 996/2014] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:35 2022] DELETE /identity/v3/projects/93486daa515540e7aa5a340f60898f89 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:36.102442 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7dc72-5a95-4805-bccf-869cc0b51e26 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:36.102964 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7dc72-5a95-4805-bccf-869cc0b51e26 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:36.103043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7dc72-5a95-4805-bccf-869cc0b51e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:36.141186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-d5d7dc72-5a95-4805-bccf-869cc0b51e26 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:36.142430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d5d7dc72-5a95-4805-bccf-869cc0b51e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:36.292467 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1019/2015] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:36.304810 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d03113-4a74-4260-b49d-606c2bfd26ee None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:36.305218 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d03113-4a74-4260-b49d-606c2bfd26ee None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:36.306571 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d03113-4a74-4260-b49d-606c2bfd26ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:36.352910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-09d03113-4a74-4260-b49d-606c2bfd26ee None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:36.356285 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-09d03113-4a74-4260-b49d-606c2bfd26ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:36.379426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 997/2016] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:36.382398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7dc72-5a95-4805-bccf-869cc0b51e26 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:36.396421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7dc72-5a95-4805-bccf-869cc0b51e26 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:36.413714 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b9d2a-d0f2-400a-b158-9b6d276b7794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:36.414419 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b9d2a-d0f2-400a-b158-9b6d276b7794 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:36.414688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b9d2a-d0f2-400a-b158-9b6d276b7794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:36.415137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b9d2a-d0f2-400a-b158-9b6d276b7794 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:36.416027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b9d2a-d0f2-400a-b158-9b6d276b7794 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:36.416502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b9d2a-d0f2-400a-b158-9b6d276b7794 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:36.418347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b9d2a-d0f2-400a-b158-9b6d276b7794 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:36.447673 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1020/2017] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 22:06:36.451794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d03113-4a74-4260-b49d-606c2bfd26ee None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:36.469035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d03113-4a74-4260-b49d-606c2bfd26ee None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:36.474648 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e369d-1e15-4543-ac75-e5ab59ca2af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:36.475365 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e369d-1e15-4543-ac75-e5ab59ca2af0 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:36.475702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e369d-1e15-4543-ac75-e5ab59ca2af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:36.476487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e369d-1e15-4543-ac75-e5ab59ca2af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:36.477351 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e369d-1e15-4543-ac75-e5ab59ca2af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:36.479897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e369d-1e15-4543-ac75-e5ab59ca2af0 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:36.514973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 998/2018] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:36 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:36.519987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b9d2a-d0f2-400a-b158-9b6d276b7794 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:36.536659 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b9d2a-d0f2-400a-b158-9b6d276b7794 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:36.557120 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e369d-1e15-4543-ac75-e5ab59ca2af0 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:36.563579 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-444e369d-1e15-4543-ac75-e5ab59ca2af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:36.589222 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:36.641498 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f6cc5-f863-435e-848c-8b6c91ea19dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:36.642345 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f6cc5-f863-435e-848c-8b6c91ea19dc admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:36.642901 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f6cc5-f863-435e-848c-8b6c91ea19dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:36.643342 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f6cc5-f863-435e-848c-8b6c91ea19dc admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:36.644272 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f6cc5-f863-435e-848c-8b6c91ea19dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:36.647429 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f6cc5-f863-435e-848c-8b6c91ea19dc admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:36.680027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1021/2019] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:36 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:36.680027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f6cc5-f863-435e-848c-8b6c91ea19dc admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:36.680027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e80f6cc5-f863-435e-848c-8b6c91ea19dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:36.699284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f6cc5-f863-435e-848c-8b6c91ea19dc admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:36.701979 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3e60e-cf92-4ea6-ae8a-b6f0ee62ce16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:36.703151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3e60e-cf92-4ea6-ae8a-b6f0ee62ce16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:36.703891 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3e60e-cf92-4ea6-ae8a-b6f0ee62ce16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:36.704425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3e60e-cf92-4ea6-ae8a-b6f0ee62ce16 admin admin] PATH_INFO: `/v3/users/507b217f9e104f6f9c9d80541695b423` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:36.705931 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3e60e-cf92-4ea6-ae8a-b6f0ee62ce16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=507b217f9e104f6f9c9d80541695b423)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:36.708180 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3e60e-cf92-4ea6-ae8a-b6f0ee62ce16 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:36.717015 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:36.723304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e369d-1e15-4543-ac75-e5ab59ca2af0 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:36.735388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc997f-92fe-4b6a-9729-d03a0cd627a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:36.737228 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc997f-92fe-4b6a-9729-d03a0cd627a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:36.737613 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc997f-92fe-4b6a-9729-d03a0cd627a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:36.737989 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc997f-92fe-4b6a-9729-d03a0cd627a0 admin admin] PATH_INFO: `/v3/users/b5dd9e216b7344bead337a573126e46d` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:36.739378 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc997f-92fe-4b6a-9729-d03a0cd627a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5dd9e216b7344bead337a573126e46d)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:36.741694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc997f-92fe-4b6a-9729-d03a0cd627a0 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:36.791879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d1d3e60e-cf92-4ea6-ae8a-b6f0ee62ce16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '507b217f9e104f6f9c9d80541695b423'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:36.806519 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4fbc997f-92fe-4b6a-9729-d03a0cd627a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5dd9e216b7344bead337a573126e46d'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:36.815143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d1d3e60e-cf92-4ea6-ae8a-b6f0ee62ce16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '507b217f9e104f6f9c9d80541695b423'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:36.816184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4fbc997f-92fe-4b6a-9729-d03a0cd627a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5dd9e216b7344bead337a573126e46d'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:36.827208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d1d3e60e-cf92-4ea6-ae8a-b6f0ee62ce16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '507b217f9e104f6f9c9d80541695b423'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:36.828217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4fbc997f-92fe-4b6a-9729-d03a0cd627a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5dd9e216b7344bead337a573126e46d'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:36.829092 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4fbc997f-92fe-4b6a-9729-d03a0cd627a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5dd9e216b7344bead337a573126e46d'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:36.830240 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4fbc997f-92fe-4b6a-9729-d03a0cd627a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5dd9e216b7344bead337a573126e46d'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:36.837739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d1d3e60e-cf92-4ea6-ae8a-b6f0ee62ce16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '507b217f9e104f6f9c9d80541695b423'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:36.838843 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d1d3e60e-cf92-4ea6-ae8a-b6f0ee62ce16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '507b217f9e104f6f9c9d80541695b423'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:36.849619 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1022/2020] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:36 2022] DELETE /identity/v3/users/507b217f9e104f6f9c9d80541695b423 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:36.851902 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3e60e-cf92-4ea6-ae8a-b6f0ee62ce16 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:36.854260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d1d3e60e-cf92-4ea6-ae8a-b6f0ee62ce16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:36.863017 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 999/2021] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:36 2022] DELETE /identity/v3/users/b5dd9e216b7344bead337a573126e46d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:36.864427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d0475-d787-4c03-9753-1876b7ff219a None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:36.864743 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d0475-d787-4c03-9753-1876b7ff219a None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:36.865027 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d0475-d787-4c03-9753-1876b7ff219a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:36.894186 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-630d0475-d787-4c03-9753-1876b7ff219a None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:36.895612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-630d0475-d787-4c03-9753-1876b7ff219a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:36.966030 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3e60e-cf92-4ea6-ae8a-b6f0ee62ce16 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:37.001482 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1000/2022] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:37.006112 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe05ed-ca80-4372-95f5-71016ceade1d None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:37.006112 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe05ed-ca80-4372-95f5-71016ceade1d None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:37.006112 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe05ed-ca80-4372-95f5-71016ceade1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:37.034414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-eafe05ed-ca80-4372-95f5-71016ceade1d None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:37.036764 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-eafe05ed-ca80-4372-95f5-71016ceade1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:37.053051 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1001/2023] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:37.054892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe05ed-ca80-4372-95f5-71016ceade1d None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:37.061454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-eafe05ed-ca80-4372-95f5-71016ceade1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:37.064208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:37.094507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4160a-b9d2-425e-83f7-851fc22ee1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:37.095288 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4160a-b9d2-425e-83f7-851fc22ee1ee admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:37.095891 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4160a-b9d2-425e-83f7-851fc22ee1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:37.096679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4160a-b9d2-425e-83f7-851fc22ee1ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:37.097569 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4160a-b9d2-425e-83f7-851fc22ee1ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:37.100027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4160a-b9d2-425e-83f7-851fc22ee1ee admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:37.114230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1023/2024] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:37.115344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4160a-b9d2-425e-83f7-851fc22ee1ee admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:37.118416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e3a4160a-b9d2-425e-83f7-851fc22ee1ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:37.174473 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe05ed-ca80-4372-95f5-71016ceade1d None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:37.190966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcaf550-c873-42fb-a119-9235ef05e8af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:37.192344 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcaf550-c873-42fb-a119-9235ef05e8af service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:37.194272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcaf550-c873-42fb-a119-9235ef05e8af service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:37.195403 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcaf550-c873-42fb-a119-9235ef05e8af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:37.219451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcaf550-c873-42fb-a119-9235ef05e8af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:37.220121 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4160a-b9d2-425e-83f7-851fc22ee1ee admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:37.221610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcaf550-c873-42fb-a119-9235ef05e8af service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:37.231932 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb712d-c7ae-40b1-87a7-8ddfcdc662dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:37.232689 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb712d-c7ae-40b1-87a7-8ddfcdc662dc service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:37.233747 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb712d-c7ae-40b1-87a7-8ddfcdc662dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:37.234848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb712d-c7ae-40b1-87a7-8ddfcdc662dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:37.258304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb712d-c7ae-40b1-87a7-8ddfcdc662dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:37.260077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeb712d-c7ae-40b1-87a7-8ddfcdc662dc service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:37.286770 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1002/2025] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:37.289886 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1024/2026] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:37.290261 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcaf550-c873-42fb-a119-9235ef05e8af service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:37.323866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcaf550-c873-42fb-a119-9235ef05e8af service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:37.335853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835ce3a-a1af-4b50-8347-b7177fdb4cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:37.336570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835ce3a-a1af-4b50-8347-b7177fdb4cac admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:37.338538 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835ce3a-a1af-4b50-8347-b7177fdb4cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:37.341696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835ce3a-a1af-4b50-8347-b7177fdb4cac admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:37.342722 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835ce3a-a1af-4b50-8347-b7177fdb4cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:37.347944 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835ce3a-a1af-4b50-8347-b7177fdb4cac admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:37.377458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1003/2027] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:37.379106 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb712d-c7ae-40b1-87a7-8ddfcdc662dc service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:37.399548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb712d-c7ae-40b1-87a7-8ddfcdc662dc service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:37.422941 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43ac6d-d232-4f98-b06b-8011f875e78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:37.424301 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43ac6d-d232-4f98-b06b-8011f875e78a admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:37.425424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43ac6d-d232-4f98-b06b-8011f875e78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:37.426984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43ac6d-d232-4f98-b06b-8011f875e78a admin admin] PATH_INFO: `/v3/projects/eb7438b6f91542c79eb4067ff42dde07/users/a5cf8ecf682848e5950ae10e56e155ee/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:37.479424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43ac6d-d232-4f98-b06b-8011f875e78a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb7438b6f91542c79eb4067ff42dde07, user_id=a5cf8ecf682848e5950ae10e56e155ee, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:37.488955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43ac6d-d232-4f98-b06b-8011f875e78a admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:37.528940 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1025/2028] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:37 2022] PUT /identity/v3/projects/eb7438b6f91542c79eb4067ff42dde07/users/a5cf8ecf682848e5950ae10e56e155ee/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:37.543898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f90b3-1cc0-4c7a-b63e-3742be8dc180 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:37.544201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f90b3-1cc0-4c7a-b63e-3742be8dc180 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:37.544487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f90b3-1cc0-4c7a-b63e-3742be8dc180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:37.556118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:37.612571 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-5d2f90b3-1cc0-4c7a-b63e-3742be8dc180 None None] MFA Rules not processed for user `a5cf8ecf682848e5950ae10e56e155ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:37.613926 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-5d2f90b3-1cc0-4c7a-b63e-3742be8dc180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:37.798533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1004/2029] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:37.803352 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43ac6d-d232-4f98-b06b-8011f875e78a admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:37.890880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43ac6d-d232-4f98-b06b-8011f875e78a admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:37.896937 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f347d7-3d0f-4c41-8152-c49287c7bf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:37.896937 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f347d7-3d0f-4c41-8152-c49287c7bf06 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:37.897165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f347d7-3d0f-4c41-8152-c49287c7bf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:37.897344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f347d7-3d0f-4c41-8152-c49287c7bf06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:37.898206 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f347d7-3d0f-4c41-8152-c49287c7bf06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:37.900907 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f347d7-3d0f-4c41-8152-c49287c7bf06 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:37.937260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1026/2030] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:37 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:06:37.941017 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f90b3-1cc0-4c7a-b63e-3742be8dc180 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:37.953889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f90b3-1cc0-4c7a-b63e-3742be8dc180 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:38.025023 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfe5ea-87b0-4f8b-bf50-fa82c5169475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:38.025656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfe5ea-87b0-4f8b-bf50-fa82c5169475 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:38.025774 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfe5ea-87b0-4f8b-bf50-fa82c5169475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:38.026048 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfe5ea-87b0-4f8b-bf50-fa82c5169475 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:38.026778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfe5ea-87b0-4f8b-bf50-fa82c5169475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:38.028792 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfe5ea-87b0-4f8b-bf50-fa82c5169475 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:38.054129 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1005/2031] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:37 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:06:38.055411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f347d7-3d0f-4c41-8152-c49287c7bf06 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:38.065975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f347d7-3d0f-4c41-8152-c49287c7bf06 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:38.071471 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ff8a2-1b7d-4821-934f-b9de99e2a64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:38.072137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ff8a2-1b7d-4821-934f-b9de99e2a64c admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:38.072424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ff8a2-1b7d-4821-934f-b9de99e2a64c admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:38.072699 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ff8a2-1b7d-4821-934f-b9de99e2a64c admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:38.073451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ff8a2-1b7d-4821-934f-b9de99e2a64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:38.075063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ff8a2-1b7d-4821-934f-b9de99e2a64c admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:38.090144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1027/2032] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:38.091431 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfe5ea-87b0-4f8b-bf50-fa82c5169475 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:38.101757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfe5ea-87b0-4f8b-bf50-fa82c5169475 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:38.106866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fb5f1-d249-4d8d-8919-88ff2b69c2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:38.107785 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fb5f1-d249-4d8d-8919-88ff2b69c2b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:38.108057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fb5f1-d249-4d8d-8919-88ff2b69c2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:38.108339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fb5f1-d249-4d8d-8919-88ff2b69c2b8 admin admin] PATH_INFO: `/v3/projects/79be3adf1c9a43a8bc0bb9b132c80f15/users/4f40d26bb40b4b9e9804ec3d4cffce0d/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:38.128871 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2fb5f1-d249-4d8d-8919-88ff2b69c2b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79be3adf1c9a43a8bc0bb9b132c80f15, user_id=4f40d26bb40b4b9e9804ec3d4cffce0d, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:38.133402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2fb5f1-d249-4d8d-8919-88ff2b69c2b8 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:38.149092 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1006/2033] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:38 2022] PUT /identity/v3/projects/79be3adf1c9a43a8bc0bb9b132c80f15/users/4f40d26bb40b4b9e9804ec3d4cffce0d/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:38.153546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ff8a2-1b7d-4821-934f-b9de99e2a64c admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:38.163988 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ff8a2-1b7d-4821-934f-b9de99e2a64c admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:38.194036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:38.225121 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d9fb3-cfe0-4e5c-bb8b-cd26c34cec5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:38.225773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d9fb3-cfe0-4e5c-bb8b-cd26c34cec5e admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:38.226212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d9fb3-cfe0-4e5c-bb8b-cd26c34cec5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:38.226553 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d9fb3-cfe0-4e5c-bb8b-cd26c34cec5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:38.227289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d9fb3-cfe0-4e5c-bb8b-cd26c34cec5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:38.228960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d9fb3-cfe0-4e5c-bb8b-cd26c34cec5e admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:38.242520 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1028/2034] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:38.244044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fb5f1-d249-4d8d-8919-88ff2b69c2b8 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:38.255252 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fb5f1-d249-4d8d-8919-88ff2b69c2b8 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:38.261357 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b6d95-db09-46f1-97f3-08c1f78fffe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:38.262248 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b6d95-db09-46f1-97f3-08c1f78fffe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:38.262532 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b6d95-db09-46f1-97f3-08c1f78fffe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:38.262810 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b6d95-db09-46f1-97f3-08c1f78fffe5 admin admin] PATH_INFO: `/v3/projects/79be3adf1c9a43a8bc0bb9b132c80f15/users/4f40d26bb40b4b9e9804ec3d4cffce0d/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:38.264886 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b6d95-db09-46f1-97f3-08c1f78fffe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79be3adf1c9a43a8bc0bb9b132c80f15, user_id=4f40d26bb40b4b9e9804ec3d4cffce0d, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:38.267609 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b6d95-db09-46f1-97f3-08c1f78fffe5 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:38.280653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1007/2035] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:38 2022] PUT /identity/v3/projects/79be3adf1c9a43a8bc0bb9b132c80f15/users/4f40d26bb40b4b9e9804ec3d4cffce0d/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:38.290659 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bad70-ee45-4252-942a-1170dfe79508 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:38.290951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bad70-ee45-4252-942a-1170dfe79508 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:38.291232 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bad70-ee45-4252-942a-1170dfe79508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:38.318282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-882bad70-ee45-4252-942a-1170dfe79508 None None] MFA Rules not processed for user `4f40d26bb40b4b9e9804ec3d4cffce0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:38.320245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-882bad70-ee45-4252-942a-1170dfe79508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:38.397292 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1029/2036] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:38.412312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b6d95-db09-46f1-97f3-08c1f78fffe5 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:38.414221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-f92b6d95-db09-46f1-97f3-08c1f78fffe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:38.416428 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bad70-ee45-4252-942a-1170dfe79508 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:38.417757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-882bad70-ee45-4252-942a-1170dfe79508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:38.476685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:38.496900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bad70-ee45-4252-942a-1170dfe79508 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:38.501179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6ec36-1be6-4525-b329-a62206f40812 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:38.501847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6ec36-1be6-4525-b329-a62206f40812 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:38.502132 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6ec36-1be6-4525-b329-a62206f40812 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:38.502410 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6ec36-1be6-4525-b329-a62206f40812 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:38.504363 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b6d95-db09-46f1-97f3-08c1f78fffe5 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:38.510806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:38.511745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:38.512124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:38.512467 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7 admin admin] PATH_INFO: `/v3/projects/2e27bde984bc443dbb465165f0a79b28` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:38.514062 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e27bde984bc443dbb465165f0a79b28)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:38.516445 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:38.567703 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e27bde984bc443dbb465165f0a79b28', 'request_id': 'req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:38.571107 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6ec36-1be6-4525-b329-a62206f40812 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:38.572576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6ec36-1be6-4525-b329-a62206f40812 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:38.574946 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e27bde984bc443dbb465165f0a79b28', 'request_id': 'req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:38.583129 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e27bde984bc443dbb465165f0a79b28', 'request_id': 'req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:38.586186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1030/2037] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:38.587558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6ec36-1be6-4525-b329-a62206f40812 service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:38.589555 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-80e6ec36-1be6-4525-b329-a62206f40812 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:38.595267 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7 admin admin] The token cache is being invalidate because project 2e27bde984bc443dbb465165f0a79b28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:38.595561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:38.596219 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:38.602971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1008/2038] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:38 2022] DELETE /identity/v3/projects/2e27bde984bc443dbb465165f0a79b28 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:38.602971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:38.604491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:38.608235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6ec36-1be6-4525-b329-a62206f40812 service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:38.613069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:38.695128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc968bc9-5536-4e84-ac4e-f093c04c3c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:38.697285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc968bc9-5536-4e84-ac4e-f093c04c3c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:38.697953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc968bc9-5536-4e84-ac4e-f093c04c3c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:38.698420 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc968bc9-5536-4e84-ac4e-f093c04c3c99 admin admin] PATH_INFO: `/v3/projects/d9d67b22b9874e15933caef64ee8b4d3` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:38.700494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc968bc9-5536-4e84-ac4e-f093c04c3c99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9d67b22b9874e15933caef64ee8b4d3)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:38.704770 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc968bc9-5536-4e84-ac4e-f093c04c3c99 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:38.707589 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a38ea-c87f-473a-81e1-53aaf746b8d7 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:38.720253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143eadd-dffa-4942-ac85-6eedc6810462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:38.722803 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143eadd-dffa-4942-ac85-6eedc6810462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:38.725048 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143eadd-dffa-4942-ac85-6eedc6810462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:38.725581 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143eadd-dffa-4942-ac85-6eedc6810462 admin admin] PATH_INFO: `/v3/users/3977f504291a4143a235991b6dc71282` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:38.729391 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6143eadd-dffa-4942-ac85-6eedc6810462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3977f504291a4143a235991b6dc71282)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:38.733233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6143eadd-dffa-4942-ac85-6eedc6810462 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:38.808304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dc968bc9-5536-4e84-ac4e-f093c04c3c99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9d67b22b9874e15933caef64ee8b4d3', 'request_id': 'req-dc968bc9-5536-4e84-ac4e-f093c04c3c99'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:38.816375 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dc968bc9-5536-4e84-ac4e-f093c04c3c99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9d67b22b9874e15933caef64ee8b4d3', 'request_id': 'req-dc968bc9-5536-4e84-ac4e-f093c04c3c99'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:38.825951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dc968bc9-5536-4e84-ac4e-f093c04c3c99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9d67b22b9874e15933caef64ee8b4d3', 'request_id': 'req-dc968bc9-5536-4e84-ac4e-f093c04c3c99'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:38.838016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6143eadd-dffa-4942-ac85-6eedc6810462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3977f504291a4143a235991b6dc71282'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:38.840859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dc968bc9-5536-4e84-ac4e-f093c04c3c99 admin admin] The token cache is being invalidate because project d9d67b22b9874e15933caef64ee8b4d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:38.841170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dc968bc9-5536-4e84-ac4e-f093c04c3c99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:38.841925 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dc968bc9-5536-4e84-ac4e-f093c04c3c99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:38.844463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1031/2039] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:38 2022] DELETE /identity/v3/projects/d9d67b22b9874e15933caef64ee8b4d3 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:38.849159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc968bc9-5536-4e84-ac4e-f093c04c3c99 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:38.851210 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-dc968bc9-5536-4e84-ac4e-f093c04c3c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:38.858334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6143eadd-dffa-4942-ac85-6eedc6810462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3977f504291a4143a235991b6dc71282'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:38.872178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6143eadd-dffa-4942-ac85-6eedc6810462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3977f504291a4143a235991b6dc71282'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:38.874040 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6143eadd-dffa-4942-ac85-6eedc6810462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3977f504291a4143a235991b6dc71282'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:38.874913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6143eadd-dffa-4942-ac85-6eedc6810462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3977f504291a4143a235991b6dc71282'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:38.895958 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:38.909548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1009/2040] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:38 2022] DELETE /identity/v3/users/3977f504291a4143a235991b6dc71282 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:38.983584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc968bc9-5536-4e84-ac4e-f093c04c3c99 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:39.070676 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:39.092411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fdc2d-2d7d-4c32-8846-4a82bb26db82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:39.093743 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fdc2d-2d7d-4c32-8846-4a82bb26db82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:39.094322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fdc2d-2d7d-4c32-8846-4a82bb26db82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:39.094959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fdc2d-2d7d-4c32-8846-4a82bb26db82 admin admin] PATH_INFO: `/v3/users/1a3c393b00e34c3b92d4e46570b1b96a` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:39.096958 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fdc2d-2d7d-4c32-8846-4a82bb26db82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a3c393b00e34c3b92d4e46570b1b96a)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:39.099583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fdc2d-2d7d-4c32-8846-4a82bb26db82 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:39.164209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-472fdc2d-2d7d-4c32-8846-4a82bb26db82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a3c393b00e34c3b92d4e46570b1b96a'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.181633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-472fdc2d-2d7d-4c32-8846-4a82bb26db82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a3c393b00e34c3b92d4e46570b1b96a'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.189407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-472fdc2d-2d7d-4c32-8846-4a82bb26db82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a3c393b00e34c3b92d4e46570b1b96a'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.199635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-472fdc2d-2d7d-4c32-8846-4a82bb26db82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a3c393b00e34c3b92d4e46570b1b96a'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.201172 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-472fdc2d-2d7d-4c32-8846-4a82bb26db82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a3c393b00e34c3b92d4e46570b1b96a'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.208601 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1032/2041] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:38 2022] DELETE /identity/v3/users/1a3c393b00e34c3b92d4e46570b1b96a => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:39.307612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143eadd-dffa-4942-ac85-6eedc6810462 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:39.309429 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6143eadd-dffa-4942-ac85-6eedc6810462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:39.326596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143eadd-dffa-4942-ac85-6eedc6810462 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:39.399053 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b5366-05d6-4a04-9994-70cf1a341820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:39.400145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b5366-05d6-4a04-9994-70cf1a341820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:39.400408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b5366-05d6-4a04-9994-70cf1a341820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:39.400825 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b5366-05d6-4a04-9994-70cf1a341820 admin admin] PATH_INFO: `/v3/projects/71cbf4330a98405da75453640e6f2a41` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:39.402039 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b5366-05d6-4a04-9994-70cf1a341820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71cbf4330a98405da75453640e6f2a41)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:39.404193 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b5366-05d6-4a04-9994-70cf1a341820 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:39.474672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-066b5366-05d6-4a04-9994-70cf1a341820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71cbf4330a98405da75453640e6f2a41', 'request_id': 'req-066b5366-05d6-4a04-9994-70cf1a341820'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.481474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-066b5366-05d6-4a04-9994-70cf1a341820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71cbf4330a98405da75453640e6f2a41', 'request_id': 'req-066b5366-05d6-4a04-9994-70cf1a341820'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.490775 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-066b5366-05d6-4a04-9994-70cf1a341820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71cbf4330a98405da75453640e6f2a41', 'request_id': 'req-066b5366-05d6-4a04-9994-70cf1a341820'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.506654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-066b5366-05d6-4a04-9994-70cf1a341820 admin admin] The token cache is being invalidate because project 71cbf4330a98405da75453640e6f2a41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:39.508134 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-066b5366-05d6-4a04-9994-70cf1a341820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.508415 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-066b5366-05d6-4a04-9994-70cf1a341820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.510142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1010/2042] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:39 2022] DELETE /identity/v3/projects/71cbf4330a98405da75453640e6f2a41 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:39.514350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fdc2d-2d7d-4c32-8846-4a82bb26db82 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:39.516123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-472fdc2d-2d7d-4c32-8846-4a82bb26db82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:39.531559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fdc2d-2d7d-4c32-8846-4a82bb26db82 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:39.551193 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b5366-05d6-4a04-9994-70cf1a341820 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:39.552684 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-066b5366-05d6-4a04-9994-70cf1a341820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:39.614453 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c9b95-718c-484a-94ba-60a7aeae09f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:39.615558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c9b95-718c-484a-94ba-60a7aeae09f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:39.615927 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c9b95-718c-484a-94ba-60a7aeae09f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:39.616203 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c9b95-718c-484a-94ba-60a7aeae09f2 admin admin] PATH_INFO: `/v3/users/99137ccf6dc845c696c276c5f1d9e4cf` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:39.618555 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c9b95-718c-484a-94ba-60a7aeae09f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99137ccf6dc845c696c276c5f1d9e4cf)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:39.620742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c9b95-718c-484a-94ba-60a7aeae09f2 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:39.650470 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b5366-05d6-4a04-9994-70cf1a341820 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:39.657865 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77f626-6c97-41bb-b466-37d64d7bf350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:39.659068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77f626-6c97-41bb-b466-37d64d7bf350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:39.659646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77f626-6c97-41bb-b466-37d64d7bf350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:39.660058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77f626-6c97-41bb-b466-37d64d7bf350 admin admin] PATH_INFO: `/v3/projects/fd58b84b67ae46dd9790ee9314e404b1` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:39.661942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77f626-6c97-41bb-b466-37d64d7bf350 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd58b84b67ae46dd9790ee9314e404b1)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:39.665055 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77f626-6c97-41bb-b466-37d64d7bf350 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:39.684980 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-b95c9b95-718c-484a-94ba-60a7aeae09f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99137ccf6dc845c696c276c5f1d9e4cf'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.701503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-b95c9b95-718c-484a-94ba-60a7aeae09f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99137ccf6dc845c696c276c5f1d9e4cf'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.702900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:39.708976 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:39.712168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-b95c9b95-718c-484a-94ba-60a7aeae09f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99137ccf6dc845c696c276c5f1d9e4cf'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.724010 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-b95c9b95-718c-484a-94ba-60a7aeae09f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99137ccf6dc845c696c276c5f1d9e4cf'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.725057 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-b95c9b95-718c-484a-94ba-60a7aeae09f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99137ccf6dc845c696c276c5f1d9e4cf'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.734298 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1033/2043] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:39 2022] DELETE /identity/v3/users/99137ccf6dc845c696c276c5f1d9e4cf => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:39.746539 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9c77f626-6c97-41bb-b466-37d64d7bf350 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd58b84b67ae46dd9790ee9314e404b1', 'request_id': 'req-9c77f626-6c97-41bb-b466-37d64d7bf350'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.753239 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9c77f626-6c97-41bb-b466-37d64d7bf350 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd58b84b67ae46dd9790ee9314e404b1', 'request_id': 'req-9c77f626-6c97-41bb-b466-37d64d7bf350'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.762551 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9c77f626-6c97-41bb-b466-37d64d7bf350 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd58b84b67ae46dd9790ee9314e404b1', 'request_id': 'req-9c77f626-6c97-41bb-b466-37d64d7bf350'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.775564 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9c77f626-6c97-41bb-b466-37d64d7bf350 admin admin] The token cache is being invalidate because project fd58b84b67ae46dd9790ee9314e404b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:39.775886 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9c77f626-6c97-41bb-b466-37d64d7bf350 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.776617 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9c77f626-6c97-41bb-b466-37d64d7bf350 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:39.778916 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1011/2044] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:39 2022] DELETE /identity/v3/projects/fd58b84b67ae46dd9790ee9314e404b1 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:39.785056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd88ef-5c85-4b65-b952-221ce193c2bb None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:39.785385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd88ef-5c85-4b65-b952-221ce193c2bb None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:39.785629 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd88ef-5c85-4b65-b952-221ce193c2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:39.813727 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-affd88ef-5c85-4b65-b952-221ce193c2bb None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:39.815699 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-affd88ef-5c85-4b65-b952-221ce193c2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:39.908758 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1034/2045] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:39.916436 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c80cd-6233-4f65-af63-b4e5d4016f0f None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:39.917038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c80cd-6233-4f65-af63-b4e5d4016f0f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:39.917641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c80cd-6233-4f65-af63-b4e5d4016f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:39.942914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-5d8c80cd-6233-4f65-af63-b4e5d4016f0f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:39.946451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-5d8c80cd-6233-4f65-af63-b4e5d4016f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:39.956094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1012/2046] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:39.961979 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd88ef-5c85-4b65-b952-221ce193c2bb None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:39.973683 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd88ef-5c85-4b65-b952-221ce193c2bb None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:39.979978 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36d410-2015-4c49-91cd-688f1ab0a628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:39.981060 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36d410-2015-4c49-91cd-688f1ab0a628 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:39.981726 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36d410-2015-4c49-91cd-688f1ab0a628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:39.982108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36d410-2015-4c49-91cd-688f1ab0a628 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:39.982974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36d410-2015-4c49-91cd-688f1ab0a628 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:39.983433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36d410-2015-4c49-91cd-688f1ab0a628 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:39.985448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36d410-2015-4c49-91cd-688f1ab0a628 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:40.003698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1035/2047] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:06:40.005157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c80cd-6233-4f65-af63-b4e5d4016f0f None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.015671 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c80cd-6233-4f65-af63-b4e5d4016f0f None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.020984 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e709a7-0039-433c-8aa9-4f878d0b6b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.021663 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e709a7-0039-433c-8aa9-4f878d0b6b24 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.021940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e709a7-0039-433c-8aa9-4f878d0b6b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.022223 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e709a7-0039-433c-8aa9-4f878d0b6b24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.023863 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e709a7-0039-433c-8aa9-4f878d0b6b24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:40.026059 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e709a7-0039-433c-8aa9-4f878d0b6b24 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:40.079213 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1013/2048] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:40 2022] POST /identity/v3/projects => generated 381 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:40.083472 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36d410-2015-4c49-91cd-688f1ab0a628 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.107910 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36d410-2015-4c49-91cd-688f1ab0a628 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.169885 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e709a7-0039-433c-8aa9-4f878d0b6b24 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.171797 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c3e709a7-0039-433c-8aa9-4f878d0b6b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:40.185598 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2398c-634d-46f6-b45b-bc26baaecd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.186310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2398c-634d-46f6-b45b-bc26baaecd92 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.186667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2398c-634d-46f6-b45b-bc26baaecd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.186870 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2398c-634d-46f6-b45b-bc26baaecd92 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.187583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2398c-634d-46f6-b45b-bc26baaecd92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:40.189692 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2398c-634d-46f6-b45b-bc26baaecd92 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:40.215307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1036/2049] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:40 2022] POST /identity/v3/users => generated 291 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:06:40.220311 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2398c-634d-46f6-b45b-bc26baaecd92 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.234466 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2398c-634d-46f6-b45b-bc26baaecd92 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.240188 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca6454-8a57-4720-a385-99c374da9795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.241061 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca6454-8a57-4720-a385-99c374da9795 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.241887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca6454-8a57-4720-a385-99c374da9795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.242178 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca6454-8a57-4720-a385-99c374da9795 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.242923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca6454-8a57-4720-a385-99c374da9795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:40.244695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca6454-8a57-4720-a385-99c374da9795 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:40.255750 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1037/2050] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:40.257835 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e709a7-0039-433c-8aa9-4f878d0b6b24 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.260849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca6454-8a57-4720-a385-99c374da9795 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.263079 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df8f5c-d678-49ce-b660-cb24b2c5d368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.264036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df8f5c-d678-49ce-b660-cb24b2c5d368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.264393 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df8f5c-d678-49ce-b660-cb24b2c5d368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.264768 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df8f5c-d678-49ce-b660-cb24b2c5d368 admin admin] PATH_INFO: `/v3/projects/e6b1c88d982b47918df97b0392f18a74` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.266044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df8f5c-d678-49ce-b660-cb24b2c5d368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6b1c88d982b47918df97b0392f18a74)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:40.268145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df8f5c-d678-49ce-b660-cb24b2c5d368 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:40.271429 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca6454-8a57-4720-a385-99c374da9795 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.277159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725fbe8-023d-45e2-949a-9f55b47bc8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.277846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725fbe8-023d-45e2-949a-9f55b47bc8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.278273 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725fbe8-023d-45e2-949a-9f55b47bc8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.278559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725fbe8-023d-45e2-949a-9f55b47bc8c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.279296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725fbe8-023d-45e2-949a-9f55b47bc8c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:40.281012 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725fbe8-023d-45e2-949a-9f55b47bc8c2 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:40.293898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1038/2051] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:40.298992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725fbe8-023d-45e2-949a-9f55b47bc8c2 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.310782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725fbe8-023d-45e2-949a-9f55b47bc8c2 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.322528 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f5df8f5c-d678-49ce-b660-cb24b2c5d368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6b1c88d982b47918df97b0392f18a74', 'request_id': 'req-f5df8f5c-d678-49ce-b660-cb24b2c5d368'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:40.329465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f5df8f5c-d678-49ce-b660-cb24b2c5d368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6b1c88d982b47918df97b0392f18a74', 'request_id': 'req-f5df8f5c-d678-49ce-b660-cb24b2c5d368'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:40.338418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f5df8f5c-d678-49ce-b660-cb24b2c5d368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6b1c88d982b47918df97b0392f18a74', 'request_id': 'req-f5df8f5c-d678-49ce-b660-cb24b2c5d368'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:40.351367 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f5df8f5c-d678-49ce-b660-cb24b2c5d368 admin admin] The token cache is being invalidate because project e6b1c88d982b47918df97b0392f18a74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:40.351726 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f5df8f5c-d678-49ce-b660-cb24b2c5d368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:40.352670 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:40.353731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f5df8f5c-d678-49ce-b660-cb24b2c5d368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:40.355783 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1014/2052] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:40 2022] DELETE /identity/v3/projects/e6b1c88d982b47918df97b0392f18a74 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:40.361683 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe448c2b-c70f-47e2-bc03-3e41c5f07160 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.362046 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe448c2b-c70f-47e2-bc03-3e41c5f07160 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.362426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe448c2b-c70f-47e2-bc03-3e41c5f07160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.382905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9513dd-4ece-4472-92f9-a36f98006296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.384763 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-fe448c2b-c70f-47e2-bc03-3e41c5f07160 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:40.384918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9513dd-4ece-4472-92f9-a36f98006296 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.384918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9513dd-4ece-4472-92f9-a36f98006296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.384918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9513dd-4ece-4472-92f9-a36f98006296 admin admin] PATH_INFO: `/v3/projects/baf2fe80d12e4167a6db028100ebaa8f/users/6c243576bc0d472c94a263c15e5f4942/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.385185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-fe448c2b-c70f-47e2-bc03-3e41c5f07160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:40.400280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1015/2053] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:40.401915 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9513dd-4ece-4472-92f9-a36f98006296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baf2fe80d12e4167a6db028100ebaa8f, user_id=6c243576bc0d472c94a263c15e5f4942, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:40.402054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b2923-dac0-447d-86d4-211e2d0e2b72 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.402261 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b2923-dac0-447d-86d4-211e2d0e2b72 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.402543 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b2923-dac0-447d-86d4-211e2d0e2b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.404684 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9513dd-4ece-4472-92f9-a36f98006296 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:40.420446 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1039/2054] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:40 2022] PUT /identity/v3/projects/baf2fe80d12e4167a6db028100ebaa8f/users/6c243576bc0d472c94a263c15e5f4942/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:40.425392 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-204b2923-dac0-447d-86d4-211e2d0e2b72 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:40.426856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-204b2923-dac0-447d-86d4-211e2d0e2b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:40.434793 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c09c5-262e-4e95-99f4-56b89dae44d5 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.435160 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c09c5-262e-4e95-99f4-56b89dae44d5 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.435278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:40.435511 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c09c5-262e-4e95-99f4-56b89dae44d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.465505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-3f2c09c5-262e-4e95-99f4-56b89dae44d5 None None] MFA Rules not processed for user `6c243576bc0d472c94a263c15e5f4942`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:40.467077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-3f2c09c5-262e-4e95-99f4-56b89dae44d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:40.499250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1016/2055] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:40.503624 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b2923-dac0-447d-86d4-211e2d0e2b72 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.513976 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b2923-dac0-447d-86d4-211e2d0e2b72 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.519675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62bd0d7-dd3a-4d9a-9996-72c84545a608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.519834 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bd0d7-dd3a-4d9a-9996-72c84545a608 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.520084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bd0d7-dd3a-4d9a-9996-72c84545a608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.520346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bd0d7-dd3a-4d9a-9996-72c84545a608 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.521155 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62bd0d7-dd3a-4d9a-9996-72c84545a608 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:40.522012 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62bd0d7-dd3a-4d9a-9996-72c84545a608 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:40.523576 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62bd0d7-dd3a-4d9a-9996-72c84545a608 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:40.540565 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1017/2056] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:06:40.541580 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62bd0d7-dd3a-4d9a-9996-72c84545a608 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.551835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1040/2057] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:40.552507 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62bd0d7-dd3a-4d9a-9996-72c84545a608 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.557790 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c09c5-262e-4e95-99f4-56b89dae44d5 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.559788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-3f2c09c5-262e-4e95-99f4-56b89dae44d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:40.560621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a96eb-e16c-4dd2-802c-a78001b0e503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.561404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a96eb-e16c-4dd2-802c-a78001b0e503 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.561856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a96eb-e16c-4dd2-802c-a78001b0e503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.562108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a96eb-e16c-4dd2-802c-a78001b0e503 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.562942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a96eb-e16c-4dd2-802c-a78001b0e503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:40.564959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a96eb-e16c-4dd2-802c-a78001b0e503 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:40.576847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c09c5-262e-4e95-99f4-56b89dae44d5 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.582113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454cc30-63e5-4981-8d61-3ded1a9389ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.583069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454cc30-63e5-4981-8d61-3ded1a9389ad admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.583387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454cc30-63e5-4981-8d61-3ded1a9389ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.583699 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454cc30-63e5-4981-8d61-3ded1a9389ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.584487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c454cc30-63e5-4981-8d61-3ded1a9389ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:40.586891 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c454cc30-63e5-4981-8d61-3ded1a9389ad admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:40.595417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1018/2058] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:40 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:40.599463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a96eb-e16c-4dd2-802c-a78001b0e503 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.619503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1041/2059] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:40 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:06:40.623773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454cc30-63e5-4981-8d61-3ded1a9389ad admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.676176 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a96eb-e16c-4dd2-802c-a78001b0e503 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.701849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454cc30-63e5-4981-8d61-3ded1a9389ad admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.706763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2313d-de68-4bd0-af4b-197b839e8171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.707546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2313d-de68-4bd0-af4b-197b839e8171 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.707845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2313d-de68-4bd0-af4b-197b839e8171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.708130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2313d-de68-4bd0-af4b-197b839e8171 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.708876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba2313d-de68-4bd0-af4b-197b839e8171 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:40.711477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba2313d-de68-4bd0-af4b-197b839e8171 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:40.738701 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1042/2060] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:40 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:06:40.739755 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2313d-de68-4bd0-af4b-197b839e8171 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.749507 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc249d-072c-423a-b7bf-68de8f87ddd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.750007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2313d-de68-4bd0-af4b-197b839e8171 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.750355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc249d-072c-423a-b7bf-68de8f87ddd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.750754 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc249d-072c-423a-b7bf-68de8f87ddd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.751171 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc249d-072c-423a-b7bf-68de8f87ddd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.752166 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc249d-072c-423a-b7bf-68de8f87ddd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:40.754345 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc249d-072c-423a-b7bf-68de8f87ddd0 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:40.754995 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91233f-f134-4544-bc06-a2c58864b9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.755647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91233f-f134-4544-bc06-a2c58864b9e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.755938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91233f-f134-4544-bc06-a2c58864b9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.756223 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91233f-f134-4544-bc06-a2c58864b9e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.757096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91233f-f134-4544-bc06-a2c58864b9e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:40.758812 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91233f-f134-4544-bc06-a2c58864b9e3 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:40.769488 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1043/2061] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:40.774650 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91233f-f134-4544-bc06-a2c58864b9e3 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.780875 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1019/2062] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:40 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:06:40.785199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc249d-072c-423a-b7bf-68de8f87ddd0 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.790135 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91233f-f134-4544-bc06-a2c58864b9e3 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.794789 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cf1e5-cf4d-41fc-b900-c30e582490a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.795666 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cf1e5-cf4d-41fc-b900-c30e582490a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.795942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cf1e5-cf4d-41fc-b900-c30e582490a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.796216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cf1e5-cf4d-41fc-b900-c30e582490a5 admin admin] PATH_INFO: `/v3/projects/d46f975330bf411597aa0f5fdc748440/users/10a7c19df87142db99b6d905e9b8f8f4/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.797295 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc249d-072c-423a-b7bf-68de8f87ddd0 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.798579 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:40.802224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe994f45-f5f3-4a1b-8d62-3bcaf45bd2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.802938 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe994f45-f5f3-4a1b-8d62-3bcaf45bd2bd admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.803244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe994f45-f5f3-4a1b-8d62-3bcaf45bd2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.803524 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe994f45-f5f3-4a1b-8d62-3bcaf45bd2bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.804384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe994f45-f5f3-4a1b-8d62-3bcaf45bd2bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:40.806657 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe994f45-f5f3-4a1b-8d62-3bcaf45bd2bd admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:40.814800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:40.815782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cf1e5-cf4d-41fc-b900-c30e582490a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d46f975330bf411597aa0f5fdc748440, user_id=10a7c19df87142db99b6d905e9b8f8f4, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:40.818503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cf1e5-cf4d-41fc-b900-c30e582490a5 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:40.823643 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1020/2063] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:06:40.825014 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe994f45-f5f3-4a1b-8d62-3bcaf45bd2bd admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.837138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe994f45-f5f3-4a1b-8d62-3bcaf45bd2bd admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.838111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1044/2064] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:40 2022] PUT /identity/v3/projects/d46f975330bf411597aa0f5fdc748440/users/10a7c19df87142db99b6d905e9b8f8f4/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:40.846889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cf1e5-cf4d-41fc-b900-c30e582490a5 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.921276 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bb2de-7fa9-47be-8711-dda5a1104f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.922046 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bb2de-7fa9-47be-8711-dda5a1104f2a admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.922236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bb2de-7fa9-47be-8711-dda5a1104f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.923228 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bb2de-7fa9-47be-8711-dda5a1104f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.924296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1bb2de-7fa9-47be-8711-dda5a1104f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:40.926041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1bb2de-7fa9-47be-8711-dda5a1104f2a admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:40.936883 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cf1e5-cf4d-41fc-b900-c30e582490a5 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.951740 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4256a-deb4-4e6c-a10d-de246005384c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.951740 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4256a-deb4-4e6c-a10d-de246005384c admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.951740 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4256a-deb4-4e6c-a10d-de246005384c admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.951740 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4256a-deb4-4e6c-a10d-de246005384c admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.952106 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4256a-deb4-4e6c-a10d-de246005384c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:40.952967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4256a-deb4-4e6c-a10d-de246005384c admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:40.953866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1021/2065] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:40.962167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bb2de-7fa9-47be-8711-dda5a1104f2a admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.973048 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bb2de-7fa9-47be-8711-dda5a1104f2a admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.974834 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1045/2066] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:06:40.979264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4256a-deb4-4e6c-a10d-de246005384c admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:40.982927 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f74f8-9cbb-435b-af6a-7c4ddcfe54b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.983822 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f74f8-9cbb-435b-af6a-7c4ddcfe54b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.984101 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f74f8-9cbb-435b-af6a-7c4ddcfe54b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.984373 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f74f8-9cbb-435b-af6a-7c4ddcfe54b9 admin admin] PATH_INFO: `/v3/projects/aca050611c2041b4b0c27e10489de1d3/users/7b2d7f86f8ca4fe7aad23777c4130d61/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.989977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4256a-deb4-4e6c-a10d-de246005384c admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:40.996021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40787e6e-dfdd-42e7-86df-df57ab3b890b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:40.997011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40787e6e-dfdd-42e7-86df-df57ab3b890b admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:40.997396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40787e6e-dfdd-42e7-86df-df57ab3b890b admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:40.997643 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40787e6e-dfdd-42e7-86df-df57ab3b890b admin admin] PATH_INFO: `/v3/projects/d46f975330bf411597aa0f5fdc748440/users/10a7c19df87142db99b6d905e9b8f8f4/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:40.999865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40787e6e-dfdd-42e7-86df-df57ab3b890b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d46f975330bf411597aa0f5fdc748440, user_id=10a7c19df87142db99b6d905e9b8f8f4, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:41.002867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40787e6e-dfdd-42e7-86df-df57ab3b890b admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:41.003938 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f74f8-9cbb-435b-af6a-7c4ddcfe54b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aca050611c2041b4b0c27e10489de1d3, user_id=7b2d7f86f8ca4fe7aad23777c4130d61, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:41.006304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f74f8-9cbb-435b-af6a-7c4ddcfe54b9 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:41.016629 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1046/2067] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:40 2022] PUT /identity/v3/projects/d46f975330bf411597aa0f5fdc748440/users/10a7c19df87142db99b6d905e9b8f8f4/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:41.024939 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1022/2068] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:40 2022] PUT /identity/v3/projects/aca050611c2041b4b0c27e10489de1d3/users/7b2d7f86f8ca4fe7aad23777c4130d61/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:41.028255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc138e0-b9e5-49a4-9eeb-dc268a3d6b2a None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:41.028872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc138e0-b9e5-49a4-9eeb-dc268a3d6b2a None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:41.029507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc138e0-b9e5-49a4-9eeb-dc268a3d6b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:41.041529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b368af-5f69-43bb-9d03-f89bcf2abe97 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:41.042097 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b368af-5f69-43bb-9d03-f89bcf2abe97 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:41.042589 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b368af-5f69-43bb-9d03-f89bcf2abe97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:41.063738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-dbc138e0-b9e5-49a4-9eeb-dc268a3d6b2a None None] MFA Rules not processed for user `10a7c19df87142db99b6d905e9b8f8f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:41.064980 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-dbc138e0-b9e5-49a4-9eeb-dc268a3d6b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:41.077572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-78b368af-5f69-43bb-9d03-f89bcf2abe97 None None] MFA Rules not processed for user `7b2d7f86f8ca4fe7aad23777c4130d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:41.079214 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-78b368af-5f69-43bb-9d03-f89bcf2abe97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:41.158166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1047/2069] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:41.167487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1023/2070] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:41.171942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc138e0-b9e5-49a4-9eeb-dc268a3d6b2a None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:41.201533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b368af-5f69-43bb-9d03-f89bcf2abe97 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:41.201533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-78b368af-5f69-43bb-9d03-f89bcf2abe97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:41.245686 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:41.262322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc138e0-b9e5-49a4-9eeb-dc268a3d6b2a None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:41.267511 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452dbf1-2ffd-4a06-8f7f-3b27a4940493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:41.268168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452dbf1-2ffd-4a06-8f7f-3b27a4940493 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:41.268477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452dbf1-2ffd-4a06-8f7f-3b27a4940493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:41.268803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452dbf1-2ffd-4a06-8f7f-3b27a4940493 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:41.269103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b368af-5f69-43bb-9d03-f89bcf2abe97 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:41.269508 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452dbf1-2ffd-4a06-8f7f-3b27a4940493 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:41.271463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452dbf1-2ffd-4a06-8f7f-3b27a4940493 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:41.273071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3a969-95de-4945-be6e-780c1ef03a31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:41.273795 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3a969-95de-4945-be6e-780c1ef03a31 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:41.274151 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3a969-95de-4945-be6e-780c1ef03a31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:41.274499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3a969-95de-4945-be6e-780c1ef03a31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:41.284321 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3a969-95de-4945-be6e-780c1ef03a31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:41.286440 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3a969-95de-4945-be6e-780c1ef03a31 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:41.307559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1048/2071] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:41 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:06:41.308896 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1024/2072] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:41.311325 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452dbf1-2ffd-4a06-8f7f-3b27a4940493 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:41.381778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3a969-95de-4945-be6e-780c1ef03a31 service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:41.388055 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:41.393003 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3a969-95de-4945-be6e-780c1ef03a31 service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:41.394292 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452dbf1-2ffd-4a06-8f7f-3b27a4940493 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:41.399972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440df29-110c-4095-9761-54b77d8661c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:41.400773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440df29-110c-4095-9761-54b77d8661c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:41.401037 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440df29-110c-4095-9761-54b77d8661c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:41.401227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440df29-110c-4095-9761-54b77d8661c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:41.401921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9440df29-110c-4095-9761-54b77d8661c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:41.403824 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9440df29-110c-4095-9761-54b77d8661c9 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:41.427863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1049/2073] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:41 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:41.429294 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440df29-110c-4095-9761-54b77d8661c9 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:41.439792 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440df29-110c-4095-9761-54b77d8661c9 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:41.444992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a84f94-f616-4622-9b48-e770c90b94c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:41.445661 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a84f94-f616-4622-9b48-e770c90b94c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:41.446148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a84f94-f616-4622-9b48-e770c90b94c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:41.446429 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a84f94-f616-4622-9b48-e770c90b94c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:41.447165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a84f94-f616-4622-9b48-e770c90b94c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:41.448908 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a84f94-f616-4622-9b48-e770c90b94c6 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:41.456981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fbd69-7ac5-4cd6-9cee-4501ca6c8b2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:41.457644 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fbd69-7ac5-4cd6-9cee-4501ca6c8b2f service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:41.457962 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fbd69-7ac5-4cd6-9cee-4501ca6c8b2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:41.458545 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fbd69-7ac5-4cd6-9cee-4501ca6c8b2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:41.460963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1050/2074] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:41.465608 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a84f94-f616-4622-9b48-e770c90b94c6 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:41.487231 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a84f94-f616-4622-9b48-e770c90b94c6 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:41.492169 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19415a-e9c4-441b-aed6-505345e46a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:41.493311 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19415a-e9c4-441b-aed6-505345e46a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:41.493426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19415a-e9c4-441b-aed6-505345e46a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:41.493628 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19415a-e9c4-441b-aed6-505345e46a72 admin admin] PATH_INFO: `/v3/projects/cb628c347add44b88ac3beecdac5bcfe/users/c8e26e235f9348a1ac4b30ad608d71b4/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:41.518239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19415a-e9c4-441b-aed6-505345e46a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb628c347add44b88ac3beecdac5bcfe, user_id=c8e26e235f9348a1ac4b30ad608d71b4, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:41.520953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19415a-e9c4-441b-aed6-505345e46a72 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:41.534223 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fbd69-7ac5-4cd6-9cee-4501ca6c8b2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:41.535675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fbd69-7ac5-4cd6-9cee-4501ca6c8b2f service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:41.540215 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1051/2075] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:41 2022] PUT /identity/v3/projects/cb628c347add44b88ac3beecdac5bcfe/users/c8e26e235f9348a1ac4b30ad608d71b4/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:41.543871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19415a-e9c4-441b-aed6-505345e46a72 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:41.551207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1025/2076] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:06:41.612471 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19415a-e9c4-441b-aed6-505345e46a72 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:41.621104 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa54e9-912b-476d-a63b-98be052f679a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:41.621104 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa54e9-912b-476d-a63b-98be052f679a admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:41.621104 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa54e9-912b-476d-a63b-98be052f679a admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:41.621104 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa54e9-912b-476d-a63b-98be052f679a admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:41.621439 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fa54e9-912b-476d-a63b-98be052f679a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:41.621714 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fa54e9-912b-476d-a63b-98be052f679a admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:41.632132 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1052/2077] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:06:41.640565 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fbd69-7ac5-4cd6-9cee-4501ca6c8b2f service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:41.648978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fbd69-7ac5-4cd6-9cee-4501ca6c8b2f service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:41.653677 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c2892-09a7-487b-895a-1d6be0a62938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:41.654577 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2892-09a7-487b-895a-1d6be0a62938 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:41.654815 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2892-09a7-487b-895a-1d6be0a62938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:41.655092 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2892-09a7-487b-895a-1d6be0a62938 admin admin] PATH_INFO: `/v3/projects/cb628c347add44b88ac3beecdac5bcfe/users/c8e26e235f9348a1ac4b30ad608d71b4/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:41.657103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c2892-09a7-487b-895a-1d6be0a62938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb628c347add44b88ac3beecdac5bcfe, user_id=c8e26e235f9348a1ac4b30ad608d71b4, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:41.659764 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c2892-09a7-487b-895a-1d6be0a62938 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:41.670997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1026/2078] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:41 2022] PUT /identity/v3/projects/cb628c347add44b88ac3beecdac5bcfe/users/c8e26e235f9348a1ac4b30ad608d71b4/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:41.681991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce23fd3-e923-42f8-9797-375ad0a13e06 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:41.682387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce23fd3-e923-42f8-9797-375ad0a13e06 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:41.682811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce23fd3-e923-42f8-9797-375ad0a13e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:41.715618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-8ce23fd3-e923-42f8-9797-375ad0a13e06 None None] MFA Rules not processed for user `c8e26e235f9348a1ac4b30ad608d71b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:41.717223 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8ce23fd3-e923-42f8-9797-375ad0a13e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:41.737489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:41.804606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1053/2079] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:41.815022 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c2892-09a7-487b-895a-1d6be0a62938 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:41.823889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce23fd3-e923-42f8-9797-375ad0a13e06 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:41.879681 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c2892-09a7-487b-895a-1d6be0a62938 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:41.883650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50c737-cc1c-409b-aaed-ec7ec3f645fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:41.884316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50c737-cc1c-409b-aaed-ec7ec3f645fb service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:41.884601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50c737-cc1c-409b-aaed-ec7ec3f645fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:41.884916 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50c737-cc1c-409b-aaed-ec7ec3f645fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:41.889487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce23fd3-e923-42f8-9797-375ad0a13e06 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:41.894505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bd4d2-3d25-42fc-af4f-1c5b098be043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:41.895008 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf50c737-cc1c-409b-aaed-ec7ec3f645fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:41.895408 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bd4d2-3d25-42fc-af4f-1c5b098be043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:41.895696 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bd4d2-3d25-42fc-af4f-1c5b098be043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:41.895974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bd4d2-3d25-42fc-af4f-1c5b098be043 admin admin] PATH_INFO: `/v3/users/6c243576bc0d472c94a263c15e5f4942` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:41.896454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf50c737-cc1c-409b-aaed-ec7ec3f645fb service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:41.897225 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bd4d2-3d25-42fc-af4f-1c5b098be043 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c243576bc0d472c94a263c15e5f4942)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:41.899187 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bd4d2-3d25-42fc-af4f-1c5b098be043 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:41.906448 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1027/2080] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:41.951151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f61bd4d2-3d25-42fc-af4f-1c5b098be043 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c243576bc0d472c94a263c15e5f4942'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:41.965488 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f61bd4d2-3d25-42fc-af4f-1c5b098be043 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c243576bc0d472c94a263c15e5f4942'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:41.973358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f61bd4d2-3d25-42fc-af4f-1c5b098be043 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c243576bc0d472c94a263c15e5f4942'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:41.982623 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f61bd4d2-3d25-42fc-af4f-1c5b098be043 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c243576bc0d472c94a263c15e5f4942'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:41.983309 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f61bd4d2-3d25-42fc-af4f-1c5b098be043 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c243576bc0d472c94a263c15e5f4942'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:41.992117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1054/2081] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:41 2022] DELETE /identity/v3/users/6c243576bc0d472c94a263c15e5f4942 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:41.996483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac472237-777a-45dd-9fb4-f8c06b15c204 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:41.996833 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac472237-777a-45dd-9fb4-f8c06b15c204 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:41.997118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac472237-777a-45dd-9fb4-f8c06b15c204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:42.020001 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-ac472237-777a-45dd-9fb4-f8c06b15c204 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:42.021572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ac472237-777a-45dd-9fb4-f8c06b15c204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:42.071583 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:42.091292 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1028/2082] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:42.101625 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bd4d2-3d25-42fc-af4f-1c5b098be043 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:42.103197 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-f61bd4d2-3d25-42fc-af4f-1c5b098be043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:42.142090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:42.170179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bd4d2-3d25-42fc-af4f-1c5b098be043 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:42.174222 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef0ec2-2ade-4623-b5db-c46c900b1591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:42.175026 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef0ec2-2ade-4623-b5db-c46c900b1591 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:42.175416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef0ec2-2ade-4623-b5db-c46c900b1591 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:42.175706 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef0ec2-2ade-4623-b5db-c46c900b1591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:42.186239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef0ec2-2ade-4623-b5db-c46c900b1591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:42.187707 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef0ec2-2ade-4623-b5db-c46c900b1591 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:42.200008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1055/2083] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:42.509411 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac472237-777a-45dd-9fb4-f8c06b15c204 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:42.511331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ac472237-777a-45dd-9fb4-f8c06b15c204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:42.534297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac472237-777a-45dd-9fb4-f8c06b15c204 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:42.541820 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef0ec2-2ade-4623-b5db-c46c900b1591 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:42.543383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5cef0ec2-2ade-4623-b5db-c46c900b1591 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:42.546662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:42.548024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:42.548558 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:42.549031 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f admin admin] PATH_INFO: `/v3/projects/baf2fe80d12e4167a6db028100ebaa8f` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:42.550426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=baf2fe80d12e4167a6db028100ebaa8f)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:42.553565 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:42.558454 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef0ec2-2ade-4623-b5db-c46c900b1591 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:42.564968 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b1088-d7ad-4faa-8b73-1b35893ceb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:42.565935 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b1088-d7ad-4faa-8b73-1b35893ceb72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:42.566291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b1088-d7ad-4faa-8b73-1b35893ceb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:42.566636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b1088-d7ad-4faa-8b73-1b35893ceb72 admin admin] PATH_INFO: `/v3/users/7b2d7f86f8ca4fe7aad23777c4130d61` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:42.567933 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b1088-d7ad-4faa-8b73-1b35893ceb72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b2d7f86f8ca4fe7aad23777c4130d61)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:42.570889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b1088-d7ad-4faa-8b73-1b35893ceb72 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:42.610726 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baf2fe80d12e4167a6db028100ebaa8f', 'request_id': 'req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:42.618023 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baf2fe80d12e4167a6db028100ebaa8f', 'request_id': 'req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:42.622664 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-193b1088-d7ad-4faa-8b73-1b35893ceb72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b2d7f86f8ca4fe7aad23777c4130d61'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:42.626581 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baf2fe80d12e4167a6db028100ebaa8f', 'request_id': 'req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:42.635407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-193b1088-d7ad-4faa-8b73-1b35893ceb72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b2d7f86f8ca4fe7aad23777c4130d61'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:42.638103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f admin admin] The token cache is being invalidate because project baf2fe80d12e4167a6db028100ebaa8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:42.638432 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:42.639097 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:42.641085 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1029/2084] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:42 2022] DELETE /identity/v3/projects/baf2fe80d12e4167a6db028100ebaa8f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:42.642701 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-193b1088-d7ad-4faa-8b73-1b35893ceb72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b2d7f86f8ca4fe7aad23777c4130d61'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:42.644793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:42.646223 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:42.650452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-193b1088-d7ad-4faa-8b73-1b35893ceb72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b2d7f86f8ca4fe7aad23777c4130d61'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:42.651201 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-193b1088-d7ad-4faa-8b73-1b35893ceb72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b2d7f86f8ca4fe7aad23777c4130d61'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:42.657314 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1056/2085] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:42 2022] DELETE /identity/v3/users/7b2d7f86f8ca4fe7aad23777c4130d61 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:42.661767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631898e-c46f-4464-aaf1-8aecaf47b690 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:42.662329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631898e-c46f-4464-aaf1-8aecaf47b690 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:42.662765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631898e-c46f-4464-aaf1-8aecaf47b690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:42.682574 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-7631898e-c46f-4464-aaf1-8aecaf47b690 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:42.683819 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-7631898e-c46f-4464-aaf1-8aecaf47b690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:42.714985 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2dc4a-8025-43f5-8a30-796591e5bc7f admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:42.749264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1057/2086] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:42.760552 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631898e-c46f-4464-aaf1-8aecaf47b690 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:42.762586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-7631898e-c46f-4464-aaf1-8aecaf47b690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:42.783750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4226e-8ef4-477f-8b91-3e57bcdad82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:42.784729 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4226e-8ef4-477f-8b91-3e57bcdad82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:42.784991 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4226e-8ef4-477f-8b91-3e57bcdad82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:42.785222 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4226e-8ef4-477f-8b91-3e57bcdad82f admin admin] PATH_INFO: `/v3/users/10a7c19df87142db99b6d905e9b8f8f4` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:42.786510 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4226e-8ef4-477f-8b91-3e57bcdad82f admin admin] RBAC: Authorizing `identity:delete_user(user_id=10a7c19df87142db99b6d905e9b8f8f4)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:42.788630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4226e-8ef4-477f-8b91-3e57bcdad82f admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:42.812634 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:42.822712 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:42.828965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631898e-c46f-4464-aaf1-8aecaf47b690 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:42.833818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e1396-c6ec-4d62-a457-23498cbb5306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:42.834488 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e1396-c6ec-4d62-a457-23498cbb5306 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:42.835025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e1396-c6ec-4d62-a457-23498cbb5306 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:42.837390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e1396-c6ec-4d62-a457-23498cbb5306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:42.844985 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-55d4226e-8ef4-477f-8b91-3e57bcdad82f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10a7c19df87142db99b6d905e9b8f8f4'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:42.852269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e1396-c6ec-4d62-a457-23498cbb5306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:42.852945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-55d4226e-8ef4-477f-8b91-3e57bcdad82f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10a7c19df87142db99b6d905e9b8f8f4'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:42.853770 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e1396-c6ec-4d62-a457-23498cbb5306 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:42.861556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-55d4226e-8ef4-477f-8b91-3e57bcdad82f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10a7c19df87142db99b6d905e9b8f8f4'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:42.862315 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-55d4226e-8ef4-477f-8b91-3e57bcdad82f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10a7c19df87142db99b6d905e9b8f8f4'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:42.862998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-55d4226e-8ef4-477f-8b91-3e57bcdad82f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10a7c19df87142db99b6d905e9b8f8f4'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:42.864879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1058/2087] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:06:42.892113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1030/2088] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:42 2022] DELETE /identity/v3/users/10a7c19df87142db99b6d905e9b8f8f4 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:43.194206 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e1396-c6ec-4d62-a457-23498cbb5306 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:43.198731 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-3e0e1396-c6ec-4d62-a457-23498cbb5306 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:43.229350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e1396-c6ec-4d62-a457-23498cbb5306 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:43.266219 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4226e-8ef4-477f-8b91-3e57bcdad82f admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:43.267951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-55d4226e-8ef4-477f-8b91-3e57bcdad82f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:43.324209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:43.325510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:43.325995 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:43.326485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99 admin admin] PATH_INFO: `/v3/projects/aca050611c2041b4b0c27e10489de1d3` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:43.328533 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aca050611c2041b4b0c27e10489de1d3)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:43.330873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:43.361388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4226e-8ef4-477f-8b91-3e57bcdad82f admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:43.367238 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018f304-fb9e-443f-bad3-75c307293cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:43.368244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018f304-fb9e-443f-bad3-75c307293cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:43.368538 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018f304-fb9e-443f-bad3-75c307293cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:43.368908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018f304-fb9e-443f-bad3-75c307293cf3 admin admin] PATH_INFO: `/v3/projects/d46f975330bf411597aa0f5fdc748440` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:43.370051 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018f304-fb9e-443f-bad3-75c307293cf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d46f975330bf411597aa0f5fdc748440)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:43.371961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018f304-fb9e-443f-bad3-75c307293cf3 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:43.381455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aca050611c2041b4b0c27e10489de1d3', 'request_id': 'req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:43.387460 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aca050611c2041b4b0c27e10489de1d3', 'request_id': 'req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:43.395272 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aca050611c2041b4b0c27e10489de1d3', 'request_id': 'req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:43.407108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99 admin admin] The token cache is being invalidate because project aca050611c2041b4b0c27e10489de1d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:43.407399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:43.408047 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:43.409648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1059/2089] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:43 2022] DELETE /identity/v3/projects/aca050611c2041b4b0c27e10489de1d3 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:43.414099 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:43.415465 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:43.422113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f018f304-fb9e-443f-bad3-75c307293cf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd46f975330bf411597aa0f5fdc748440', 'request_id': 'req-f018f304-fb9e-443f-bad3-75c307293cf3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:43.428262 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f018f304-fb9e-443f-bad3-75c307293cf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd46f975330bf411597aa0f5fdc748440', 'request_id': 'req-f018f304-fb9e-443f-bad3-75c307293cf3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:43.436624 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f018f304-fb9e-443f-bad3-75c307293cf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd46f975330bf411597aa0f5fdc748440', 'request_id': 'req-f018f304-fb9e-443f-bad3-75c307293cf3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:43.449589 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f018f304-fb9e-443f-bad3-75c307293cf3 admin admin] The token cache is being invalidate because project d46f975330bf411597aa0f5fdc748440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:43.449887 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f018f304-fb9e-443f-bad3-75c307293cf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:43.450552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f018f304-fb9e-443f-bad3-75c307293cf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:43.452126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1031/2090] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:43 2022] DELETE /identity/v3/projects/d46f975330bf411597aa0f5fdc748440 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:43.456729 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf5387-450f-46c8-ab24-fc224535c788 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:43.457024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf5387-450f-46c8-ab24-fc224535c788 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:43.457307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf5387-450f-46c8-ab24-fc224535c788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:43.475195 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-f7cf5387-450f-46c8-ab24-fc224535c788 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:43.476995 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-f7cf5387-450f-46c8-ab24-fc224535c788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:43.484934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d4fd9-5181-45c5-b29e-9d00ed2ddd99 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:43.502559 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:43.512202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:43.542944 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1032/2091] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:43.550699 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973e5f7-cfe8-48f0-9bb8-5925827de666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:43.551613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973e5f7-cfe8-48f0-9bb8-5925827de666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:43.551990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973e5f7-cfe8-48f0-9bb8-5925827de666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:43.552104 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183a609-0318-4947-9fe7-6400bfcee640 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:43.552323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183a609-0318-4947-9fe7-6400bfcee640 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:43.552492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973e5f7-cfe8-48f0-9bb8-5925827de666 admin admin] PATH_INFO: `/v3/users/c8e26e235f9348a1ac4b30ad608d71b4` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:43.552654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183a609-0318-4947-9fe7-6400bfcee640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:43.554507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973e5f7-cfe8-48f0-9bb8-5925827de666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8e26e235f9348a1ac4b30ad608d71b4)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:43.556543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973e5f7-cfe8-48f0-9bb8-5925827de666 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:43.573869 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-e183a609-0318-4947-9fe7-6400bfcee640 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:43.575039 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e183a609-0318-4947-9fe7-6400bfcee640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:43.583336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1033/2092] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:43.588080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183a609-0318-4947-9fe7-6400bfcee640 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:43.598538 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183a609-0318-4947-9fe7-6400bfcee640 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:43.604424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129e3a1-7210-4405-b4e0-f7261e60bf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:43.605142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129e3a1-7210-4405-b4e0-f7261e60bf0c admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:43.605417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129e3a1-7210-4405-b4e0-f7261e60bf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:43.605696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129e3a1-7210-4405-b4e0-f7261e60bf0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:43.606432 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129e3a1-7210-4405-b4e0-f7261e60bf0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:43.606815 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129e3a1-7210-4405-b4e0-f7261e60bf0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:43.608573 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129e3a1-7210-4405-b4e0-f7261e60bf0c admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:43.613143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2973e5f7-cfe8-48f0-9bb8-5925827de666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8e26e235f9348a1ac4b30ad608d71b4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:43.624554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1034/2093] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 22:06:43.626265 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129e3a1-7210-4405-b4e0-f7261e60bf0c admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:43.631069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2973e5f7-cfe8-48f0-9bb8-5925827de666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8e26e235f9348a1ac4b30ad608d71b4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:43.636588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129e3a1-7210-4405-b4e0-f7261e60bf0c admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:43.639477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2973e5f7-cfe8-48f0-9bb8-5925827de666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8e26e235f9348a1ac4b30ad608d71b4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:43.641483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed07080-e952-44b1-a34c-efbf72901052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:43.642196 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed07080-e952-44b1-a34c-efbf72901052 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:43.642422 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed07080-e952-44b1-a34c-efbf72901052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:43.642688 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed07080-e952-44b1-a34c-efbf72901052 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:43.643426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed07080-e952-44b1-a34c-efbf72901052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:43.645396 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed07080-e952-44b1-a34c-efbf72901052 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:43.650270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2973e5f7-cfe8-48f0-9bb8-5925827de666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8e26e235f9348a1ac4b30ad608d71b4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:43.651140 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2973e5f7-cfe8-48f0-9bb8-5925827de666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8e26e235f9348a1ac4b30ad608d71b4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:43.659400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1060/2094] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:43 2022] DELETE /identity/v3/users/c8e26e235f9348a1ac4b30ad608d71b4 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:43.675825 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1035/2095] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:43 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:06:43.677057 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973e5f7-cfe8-48f0-9bb8-5925827de666 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:43.678482 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2973e5f7-cfe8-48f0-9bb8-5925827de666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:43.752520 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973e5f7-cfe8-48f0-9bb8-5925827de666 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:43.802407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:43.813711 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2137dd-18d1-4d74-bdf9-2dd275b67fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:43.814624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2137dd-18d1-4d74-bdf9-2dd275b67fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:43.815005 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2137dd-18d1-4d74-bdf9-2dd275b67fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:43.815286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2137dd-18d1-4d74-bdf9-2dd275b67fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:43.815994 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2137dd-18d1-4d74-bdf9-2dd275b67fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:43.818148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2137dd-18d1-4d74-bdf9-2dd275b67fc1 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:43.841645 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1061/2096] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:43 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:43.845793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed07080-e952-44b1-a34c-efbf72901052 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:43.857193 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed07080-e952-44b1-a34c-efbf72901052 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:43.861980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918a046-3c33-4aef-ace5-8b1f5f77e449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:43.862653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918a046-3c33-4aef-ace5-8b1f5f77e449 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:43.862934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918a046-3c33-4aef-ace5-8b1f5f77e449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:43.863206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918a046-3c33-4aef-ace5-8b1f5f77e449 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:43.863931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918a046-3c33-4aef-ace5-8b1f5f77e449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:43.865583 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918a046-3c33-4aef-ace5-8b1f5f77e449 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:43.876187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1036/2097] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:43.880596 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2137dd-18d1-4d74-bdf9-2dd275b67fc1 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:43.891570 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2137dd-18d1-4d74-bdf9-2dd275b67fc1 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:43.896549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f663b-cf1e-4a3f-817f-4ed51349a899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:43.898007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f663b-cf1e-4a3f-817f-4ed51349a899 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:43.898338 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f663b-cf1e-4a3f-817f-4ed51349a899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:43.898617 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f663b-cf1e-4a3f-817f-4ed51349a899 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:43.899350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f663b-cf1e-4a3f-817f-4ed51349a899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:43.901656 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f663b-cf1e-4a3f-817f-4ed51349a899 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:43.913581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1062/2098] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:43.917968 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918a046-3c33-4aef-ace5-8b1f5f77e449 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:43.928356 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918a046-3c33-4aef-ace5-8b1f5f77e449 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:43.933842 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda74dcf-3e39-4b25-a613-7b41cfd34224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:43.934790 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda74dcf-3e39-4b25-a613-7b41cfd34224 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:43.935488 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda74dcf-3e39-4b25-a613-7b41cfd34224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:43.935488 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda74dcf-3e39-4b25-a613-7b41cfd34224 admin admin] PATH_INFO: `/v3/projects/c53e7071c8e147ce9a5e239e773b8dcc/users/72a9a7fdb2e8491a94257215cda417b1/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:43.951398 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda74dcf-3e39-4b25-a613-7b41cfd34224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c53e7071c8e147ce9a5e239e773b8dcc, user_id=72a9a7fdb2e8491a94257215cda417b1, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:43.954210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda74dcf-3e39-4b25-a613-7b41cfd34224 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:43.960898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f663b-cf1e-4a3f-817f-4ed51349a899 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:43.963037 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-9d6f663b-cf1e-4a3f-817f-4ed51349a899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:43.982929 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f663b-cf1e-4a3f-817f-4ed51349a899 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:43.993006 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1037/2099] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:43 2022] PUT /identity/v3/projects/c53e7071c8e147ce9a5e239e773b8dcc/users/72a9a7fdb2e8491a94257215cda417b1/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:43.995058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06eaff-c408-42fd-b7f8-ba59223b9273 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:43.995419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06eaff-c408-42fd-b7f8-ba59223b9273 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:43.995765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06eaff-c408-42fd-b7f8-ba59223b9273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:44.025200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-1e06eaff-c408-42fd-b7f8-ba59223b9273 None None] MFA Rules not processed for user `72a9a7fdb2e8491a94257215cda417b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:44.026497 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1e06eaff-c408-42fd-b7f8-ba59223b9273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:44.059335 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c5111-4919-494a-b14f-4f8619bafc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:44.061010 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c5111-4919-494a-b14f-4f8619bafc09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:44.061311 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c5111-4919-494a-b14f-4f8619bafc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:44.061665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c5111-4919-494a-b14f-4f8619bafc09 admin admin] PATH_INFO: `/v3/projects/cb628c347add44b88ac3beecdac5bcfe` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:44.063216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c5111-4919-494a-b14f-4f8619bafc09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb628c347add44b88ac3beecdac5bcfe)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:44.065489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c5111-4919-494a-b14f-4f8619bafc09 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:44.108214 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1038/2100] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:44.119962 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06eaff-c408-42fd-b7f8-ba59223b9273 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:44.122259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1e06eaff-c408-42fd-b7f8-ba59223b9273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:44.123996 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-4b3c5111-4919-494a-b14f-4f8619bafc09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb628c347add44b88ac3beecdac5bcfe', 'request_id': 'req-4b3c5111-4919-494a-b14f-4f8619bafc09'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:44.130379 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-4b3c5111-4919-494a-b14f-4f8619bafc09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb628c347add44b88ac3beecdac5bcfe', 'request_id': 'req-4b3c5111-4919-494a-b14f-4f8619bafc09'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:44.140669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-4b3c5111-4919-494a-b14f-4f8619bafc09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb628c347add44b88ac3beecdac5bcfe', 'request_id': 'req-4b3c5111-4919-494a-b14f-4f8619bafc09'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:44.153845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-4b3c5111-4919-494a-b14f-4f8619bafc09 admin admin] The token cache is being invalidate because project cb628c347add44b88ac3beecdac5bcfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:44.154137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-4b3c5111-4919-494a-b14f-4f8619bafc09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:44.154810 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-4b3c5111-4919-494a-b14f-4f8619bafc09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:44.156405 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1063/2101] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:43 2022] DELETE /identity/v3/projects/cb628c347add44b88ac3beecdac5bcfe => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:44.161160 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaf2af-d8f1-4f49-885b-f3d5fa860b4d None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:44.161446 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaf2af-d8f1-4f49-885b-f3d5fa860b4d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:44.161723 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaf2af-d8f1-4f49-885b-f3d5fa860b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:44.180358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-dffaf2af-d8f1-4f49-885b-f3d5fa860b4d None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:44.181641 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-dffaf2af-d8f1-4f49-885b-f3d5fa860b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:44.185234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:44.219652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06eaff-c408-42fd-b7f8-ba59223b9273 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:44.243036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1064/2102] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:44.250631 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a02f6b-4029-44e8-acaf-f4236b092c77 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:44.250924 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a02f6b-4029-44e8-acaf-f4236b092c77 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:44.251289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a02f6b-4029-44e8-acaf-f4236b092c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:44.257469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:44.269634 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29d9a0-3e22-4162-a46e-df62012589bb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5c9ed6f9fd542069cf914003b636c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:44.270288 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29d9a0-3e22-4162-a46e-df62012589bb service glance] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:44.270575 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29d9a0-3e22-4162-a46e-df62012589bb service glance] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:44.270853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29d9a0-3e22-4162-a46e-df62012589bb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:44.270961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-95a02f6b-4029-44e8-acaf-f4236b092c77 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:44.272163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-95a02f6b-4029-44e8-acaf-f4236b092c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:44.272670 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-2b29d9a0-3e22-4162-a46e-df62012589bb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:44.280462 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1065/2103] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:44.284822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a02f6b-4029-44e8-acaf-f4236b092c77 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:44.294505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a02f6b-4029-44e8-acaf-f4236b092c77 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:44.299154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339f90f-3504-40ff-a4a2-c537e4927eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:44.299797 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339f90f-3504-40ff-a4a2-c537e4927eaa admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:44.300078 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339f90f-3504-40ff-a4a2-c537e4927eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:44.300358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339f90f-3504-40ff-a4a2-c537e4927eaa admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:44.301107 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339f90f-3504-40ff-a4a2-c537e4927eaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:44.301416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339f90f-3504-40ff-a4a2-c537e4927eaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:44.303022 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339f90f-3504-40ff-a4a2-c537e4927eaa admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:44.315714 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1066/2104] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:06:44.321539 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339f90f-3504-40ff-a4a2-c537e4927eaa admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:44.332235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339f90f-3504-40ff-a4a2-c537e4927eaa admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:44.336285 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29d9a0-3e22-4162-a46e-df62012589bb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:44.337487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3c896-66d5-48ab-8755-37cfe62e74ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:44.337807 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29d9a0-3e22-4162-a46e-df62012589bb service glance] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:44.338238 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3c896-66d5-48ab-8755-37cfe62e74ce admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:44.338559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3c896-66d5-48ab-8755-37cfe62e74ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:44.338848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3c896-66d5-48ab-8755-37cfe62e74ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:44.339844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3c896-66d5-48ab-8755-37cfe62e74ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:44.342053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3c896-66d5-48ab-8755-37cfe62e74ce admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:44.348190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1039/2105] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:44.376320 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1067/2106] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:44 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:44.377400 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29d9a0-3e22-4162-a46e-df62012589bb service glance] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:44.388742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29d9a0-3e22-4162-a46e-df62012589bb service glance] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:44.450845 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293a155-c8b6-4394-b595-81c3e88465f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:44.451544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293a155-c8b6-4394-b595-81c3e88465f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:44.451947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293a155-c8b6-4394-b595-81c3e88465f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:44.452236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293a155-c8b6-4394-b595-81c3e88465f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:44.453005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293a155-c8b6-4394-b595-81c3e88465f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:44.455009 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293a155-c8b6-4394-b595-81c3e88465f3 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:44.485280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1040/2107] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:44 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:44.486363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3c896-66d5-48ab-8755-37cfe62e74ce admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:44.497933 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3c896-66d5-48ab-8755-37cfe62e74ce admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:44.503573 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79d641-3e68-4388-afc2-9fb7af2507e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:44.504249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79d641-3e68-4388-afc2-9fb7af2507e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:44.504541 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79d641-3e68-4388-afc2-9fb7af2507e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:44.504865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79d641-3e68-4388-afc2-9fb7af2507e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:44.505604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79d641-3e68-4388-afc2-9fb7af2507e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:44.507515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79d641-3e68-4388-afc2-9fb7af2507e2 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:44.524028 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1068/2108] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:06:44.530239 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293a155-c8b6-4394-b595-81c3e88465f3 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:44.546600 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293a155-c8b6-4394-b595-81c3e88465f3 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:44.551963 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec89e8b-514d-4a3e-80e1-3b6faff789e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:44.552694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec89e8b-514d-4a3e-80e1-3b6faff789e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:44.553030 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec89e8b-514d-4a3e-80e1-3b6faff789e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:44.553392 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec89e8b-514d-4a3e-80e1-3b6faff789e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:44.554351 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec89e8b-514d-4a3e-80e1-3b6faff789e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:44.555953 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec89e8b-514d-4a3e-80e1-3b6faff789e3 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:44.572794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1041/2109] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:44.575615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79d641-3e68-4388-afc2-9fb7af2507e2 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:44.588927 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79d641-3e68-4388-afc2-9fb7af2507e2 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:44.593937 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15b0f6-c953-4c25-9246-fce406626d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:44.594874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15b0f6-c953-4c25-9246-fce406626d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:44.595159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15b0f6-c953-4c25-9246-fce406626d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:44.595442 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15b0f6-c953-4c25-9246-fce406626d16 admin admin] PATH_INFO: `/v3/projects/ebecae0398ee4e99a6b7ef9260bc39b3/users/238236d7b4284966b8989c6dea082821/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:44.611939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15b0f6-c953-4c25-9246-fce406626d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebecae0398ee4e99a6b7ef9260bc39b3, user_id=238236d7b4284966b8989c6dea082821, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:44.614681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15b0f6-c953-4c25-9246-fce406626d16 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:44.628906 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1069/2110] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:44 2022] PUT /identity/v3/projects/ebecae0398ee4e99a6b7ef9260bc39b3/users/238236d7b4284966b8989c6dea082821/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:44.640462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec466700-c695-4517-8d11-d60b767e069e None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:44.640784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec466700-c695-4517-8d11-d60b767e069e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:44.641076 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec466700-c695-4517-8d11-d60b767e069e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:44.661530 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:44.672438 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-ec466700-c695-4517-8d11-d60b767e069e None None] MFA Rules not processed for user `238236d7b4284966b8989c6dea082821`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:44.674604 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ec466700-c695-4517-8d11-d60b767e069e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:44.773939 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1042/2111] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:44.780889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15b0f6-c953-4c25-9246-fce406626d16 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:44.880194 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15b0f6-c953-4c25-9246-fce406626d16 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:44.886382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb87cbb-e877-45ea-8965-d7030ef2ab9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:44.887257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb87cbb-e877-45ea-8965-d7030ef2ab9d admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:44.887397 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb87cbb-e877-45ea-8965-d7030ef2ab9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:44.887644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb87cbb-e877-45ea-8965-d7030ef2ab9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:44.888433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb87cbb-e877-45ea-8965-d7030ef2ab9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:44.890376 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb87cbb-e877-45ea-8965-d7030ef2ab9d admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:44.930530 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1070/2112] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:44 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:44.931826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec466700-c695-4517-8d11-d60b767e069e None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:44.943150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec466700-c695-4517-8d11-d60b767e069e None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:45.002265 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66230b04-49cc-4ee1-8687-014fbe754721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:45.002993 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66230b04-49cc-4ee1-8687-014fbe754721 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:45.003315 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66230b04-49cc-4ee1-8687-014fbe754721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:45.003626 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66230b04-49cc-4ee1-8687-014fbe754721 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:45.004358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66230b04-49cc-4ee1-8687-014fbe754721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:45.006457 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66230b04-49cc-4ee1-8687-014fbe754721 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:45.027894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1043/2113] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:44 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:06:45.031895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb87cbb-e877-45ea-8965-d7030ef2ab9d admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:45.042826 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb87cbb-e877-45ea-8965-d7030ef2ab9d admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:45.048253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac68d3-88e8-4934-a811-cef700c7d333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:45.048982 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac68d3-88e8-4934-a811-cef700c7d333 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:45.049276 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac68d3-88e8-4934-a811-cef700c7d333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:45.049558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac68d3-88e8-4934-a811-cef700c7d333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:45.050291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac68d3-88e8-4934-a811-cef700c7d333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:45.051908 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac68d3-88e8-4934-a811-cef700c7d333 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:45.062840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1071/2114] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:45.067136 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66230b04-49cc-4ee1-8687-014fbe754721 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:45.078339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66230b04-49cc-4ee1-8687-014fbe754721 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:45.082895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cb845-0d06-4c3f-a4e6-2fd59645a67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:45.083762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cb845-0d06-4c3f-a4e6-2fd59645a67a admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:45.084041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cb845-0d06-4c3f-a4e6-2fd59645a67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:45.084316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cb845-0d06-4c3f-a4e6-2fd59645a67a admin admin] PATH_INFO: `/v3/projects/788b5ea0ea5745bfa44cbee3ec4f3a3e/users/19e4161ee978401995608fc53a13f4d3/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:45.101387 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6cb845-0d06-4c3f-a4e6-2fd59645a67a admin admin] RBAC: Authorizing `identity:create_grant(project_id=788b5ea0ea5745bfa44cbee3ec4f3a3e, user_id=19e4161ee978401995608fc53a13f4d3, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:45.104049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6cb845-0d06-4c3f-a4e6-2fd59645a67a admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:45.119325 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1044/2115] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:45 2022] PUT /identity/v3/projects/788b5ea0ea5745bfa44cbee3ec4f3a3e/users/19e4161ee978401995608fc53a13f4d3/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:45.123626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac68d3-88e8-4934-a811-cef700c7d333 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:45.134406 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac68d3-88e8-4934-a811-cef700c7d333 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:45.193988 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a8247-1c42-45cf-b2ca-fc8512b98499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:45.195548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a8247-1c42-45cf-b2ca-fc8512b98499 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:45.195929 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a8247-1c42-45cf-b2ca-fc8512b98499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:45.196262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a8247-1c42-45cf-b2ca-fc8512b98499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:45.197038 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a8247-1c42-45cf-b2ca-fc8512b98499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:45.198631 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3a8247-1c42-45cf-b2ca-fc8512b98499 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:45.199988 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:45.212064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1072/2116] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:45.214657 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cb845-0d06-4c3f-a4e6-2fd59645a67a admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:45.225877 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cb845-0d06-4c3f-a4e6-2fd59645a67a admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:45.230590 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2bedc-88c1-4a8c-8aeb-1b30fb754bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:45.231465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2bedc-88c1-4a8c-8aeb-1b30fb754bff admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:45.231744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2bedc-88c1-4a8c-8aeb-1b30fb754bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:45.232016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2bedc-88c1-4a8c-8aeb-1b30fb754bff admin admin] PATH_INFO: `/v3/projects/788b5ea0ea5745bfa44cbee3ec4f3a3e/users/19e4161ee978401995608fc53a13f4d3/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:45.233979 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c2bedc-88c1-4a8c-8aeb-1b30fb754bff admin admin] RBAC: Authorizing `identity:create_grant(project_id=788b5ea0ea5745bfa44cbee3ec4f3a3e, user_id=19e4161ee978401995608fc53a13f4d3, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:45.236738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c2bedc-88c1-4a8c-8aeb-1b30fb754bff admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:45.249106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1045/2117] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:45 2022] PUT /identity/v3/projects/788b5ea0ea5745bfa44cbee3ec4f3a3e/users/19e4161ee978401995608fc53a13f4d3/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:45.261372 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016d33a-26bc-44e1-b4ab-ad205bb1771d None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:45.261662 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016d33a-26bc-44e1-b4ab-ad205bb1771d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:45.261942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016d33a-26bc-44e1-b4ab-ad205bb1771d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:45.287722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-9016d33a-26bc-44e1-b4ab-ad205bb1771d None None] MFA Rules not processed for user `19e4161ee978401995608fc53a13f4d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:45.289020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-9016d33a-26bc-44e1-b4ab-ad205bb1771d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:45.356325 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1073/2118] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:45.367718 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2bedc-88c1-4a8c-8aeb-1b30fb754bff admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:45.369466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-09c2bedc-88c1-4a8c-8aeb-1b30fb754bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:45.425788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2bedc-88c1-4a8c-8aeb-1b30fb754bff admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:45.430305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6607308-675d-4ede-8a11-edcbdc8c1b2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:45.431054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6607308-675d-4ede-8a11-edcbdc8c1b2d service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:45.431427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6607308-675d-4ede-8a11-edcbdc8c1b2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:45.431829 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6607308-675d-4ede-8a11-edcbdc8c1b2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:45.437686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:45.491641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6607308-675d-4ede-8a11-edcbdc8c1b2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:45.493242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6607308-675d-4ede-8a11-edcbdc8c1b2d service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:45.502150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1046/2119] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:47.723464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016d33a-26bc-44e1-b4ab-ad205bb1771d None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:47.725150 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-9016d33a-26bc-44e1-b4ab-ad205bb1771d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:47.824616 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016d33a-26bc-44e1-b4ab-ad205bb1771d None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:47.831610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1ee28-9313-4947-9b10-d9694e9940a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:47.832778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1ee28-9313-4947-9b10-d9694e9940a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:47.832979 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1ee28-9313-4947-9b10-d9694e9940a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:47.833326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1ee28-9313-4947-9b10-d9694e9940a7 admin admin] PATH_INFO: `/v3/users/a5cf8ecf682848e5950ae10e56e155ee` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:47.834695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb1ee28-9313-4947-9b10-d9694e9940a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5cf8ecf682848e5950ae10e56e155ee)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:47.837249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb1ee28-9313-4947-9b10-d9694e9940a7 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:47.937264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-1eb1ee28-9313-4947-9b10-d9694e9940a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5cf8ecf682848e5950ae10e56e155ee'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:47.962042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6607308-675d-4ede-8a11-edcbdc8c1b2d service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:47.964377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e6607308-675d-4ede-8a11-edcbdc8c1b2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:47.966228 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-1eb1ee28-9313-4947-9b10-d9694e9940a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5cf8ecf682848e5950ae10e56e155ee'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:47.976972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-1eb1ee28-9313-4947-9b10-d9694e9940a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5cf8ecf682848e5950ae10e56e155ee'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:47.994151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-1eb1ee28-9313-4947-9b10-d9694e9940a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5cf8ecf682848e5950ae10e56e155ee'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:47.995358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-1eb1ee28-9313-4947-9b10-d9694e9940a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5cf8ecf682848e5950ae10e56e155ee'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:47.999141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:48.000573 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6607308-675d-4ede-8a11-edcbdc8c1b2d service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:48.020686 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1074/2120] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:47 2022] DELETE /identity/v3/users/a5cf8ecf682848e5950ae10e56e155ee => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:48.024388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83af47-b993-4163-a608-ba8d2a578dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:48.025784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83af47-b993-4163-a608-ba8d2a578dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:48.026050 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83af47-b993-4163-a608-ba8d2a578dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:48.026332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83af47-b993-4163-a608-ba8d2a578dce admin admin] PATH_INFO: `/v3/users/77a7cc2556ee46aca86ee8e8a86def0f` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:48.027070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba225a3a-671e-47bc-94ed-fd4572010244 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:48.027378 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba225a3a-671e-47bc-94ed-fd4572010244 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:48.027776 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba225a3a-671e-47bc-94ed-fd4572010244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:48.029989 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83af47-b993-4163-a608-ba8d2a578dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=77a7cc2556ee46aca86ee8e8a86def0f)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:48.031942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83af47-b993-4163-a608-ba8d2a578dce admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:48.089906 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-ba225a3a-671e-47bc-94ed-fd4572010244 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:48.091669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ba225a3a-671e-47bc-94ed-fd4572010244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:48.165520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8d83af47-b993-4163-a608-ba8d2a578dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77a7cc2556ee46aca86ee8e8a86def0f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:48.196435 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8d83af47-b993-4163-a608-ba8d2a578dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77a7cc2556ee46aca86ee8e8a86def0f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:48.221030 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8d83af47-b993-4163-a608-ba8d2a578dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77a7cc2556ee46aca86ee8e8a86def0f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:48.222215 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8d83af47-b993-4163-a608-ba8d2a578dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77a7cc2556ee46aca86ee8e8a86def0f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:48.224413 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8d83af47-b993-4163-a608-ba8d2a578dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77a7cc2556ee46aca86ee8e8a86def0f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:48.238390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1075/2121] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:48.250638 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba225a3a-671e-47bc-94ed-fd4572010244 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:48.252833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ba225a3a-671e-47bc-94ed-fd4572010244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:48.270109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1047/2122] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:47 2022] DELETE /identity/v3/users/77a7cc2556ee46aca86ee8e8a86def0f => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:48.275763 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bdf1f-9094-49db-b17a-08d059c739e2 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:48.276154 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bdf1f-9094-49db-b17a-08d059c739e2 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:48.276532 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bdf1f-9094-49db-b17a-08d059c739e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:48.297902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-095bdf1f-9094-49db-b17a-08d059c739e2 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:48.299194 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-095bdf1f-9094-49db-b17a-08d059c739e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:48.348460 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba225a3a-671e-47bc-94ed-fd4572010244 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:48.432768 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1048/2123] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:48.446387 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bdf1f-9094-49db-b17a-08d059c739e2 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:48.478067 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b970a-bcc2-4227-aade-02f1fa2f2be9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:48.478908 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b970a-bcc2-4227-aade-02f1fa2f2be9 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:48.479214 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b970a-bcc2-4227-aade-02f1fa2f2be9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:48.479681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b970a-bcc2-4227-aade-02f1fa2f2be9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:48.492838 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:48.501898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b970a-bcc2-4227-aade-02f1fa2f2be9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:48.503421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b970a-bcc2-4227-aade-02f1fa2f2be9 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:48.528683 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1076/2124] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:06:48.545814 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bdf1f-9094-49db-b17a-08d059c739e2 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:48.551956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fb6f3-0f56-48cf-a329-2bb4e47175d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:48.553047 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fb6f3-0f56-48cf-a329-2bb4e47175d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:48.554399 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fb6f3-0f56-48cf-a329-2bb4e47175d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:48.554805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fb6f3-0f56-48cf-a329-2bb4e47175d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:48.568292 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444fb6f3-0f56-48cf-a329-2bb4e47175d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:48.579224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444fb6f3-0f56-48cf-a329-2bb4e47175d1 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:48.623483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1049/2125] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:06:49.111388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b970a-bcc2-4227-aade-02f1fa2f2be9 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:49.112956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-6e9b970a-bcc2-4227-aade-02f1fa2f2be9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:49.128076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b970a-bcc2-4227-aade-02f1fa2f2be9 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:49.136108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfeeaf-0101-454a-86a0-da10b89cba84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:49.136903 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfeeaf-0101-454a-86a0-da10b89cba84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:49.137190 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfeeaf-0101-454a-86a0-da10b89cba84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:49.137474 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfeeaf-0101-454a-86a0-da10b89cba84 admin admin] PATH_INFO: `/v3/projects/eb7438b6f91542c79eb4067ff42dde07` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:49.144692 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfeeaf-0101-454a-86a0-da10b89cba84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb7438b6f91542c79eb4067ff42dde07)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:49.148121 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfeeaf-0101-454a-86a0-da10b89cba84 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:49.177588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fb6f3-0f56-48cf-a329-2bb4e47175d1 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:49.179721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-444fb6f3-0f56-48cf-a329-2bb4e47175d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:49.195181 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fb6f3-0f56-48cf-a329-2bb4e47175d1 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:49.200862 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf5819-64af-488c-bb0f-d4de3bd0ac10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:49.201811 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf5819-64af-488c-bb0f-d4de3bd0ac10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:49.202127 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf5819-64af-488c-bb0f-d4de3bd0ac10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:49.202414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf5819-64af-488c-bb0f-d4de3bd0ac10 admin admin] PATH_INFO: `/v3/projects/cb7e4dce11704c83a5dc447245fb8118` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:49.203660 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf5819-64af-488c-bb0f-d4de3bd0ac10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb7e4dce11704c83a5dc447245fb8118)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:49.205955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf5819-64af-488c-bb0f-d4de3bd0ac10 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:49.223715 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c5dfeeaf-0101-454a-86a0-da10b89cba84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb7438b6f91542c79eb4067ff42dde07', 'request_id': 'req-c5dfeeaf-0101-454a-86a0-da10b89cba84'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:49.228503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:49.231255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c5dfeeaf-0101-454a-86a0-da10b89cba84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb7438b6f91542c79eb4067ff42dde07', 'request_id': 'req-c5dfeeaf-0101-454a-86a0-da10b89cba84'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:49.242292 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c5dfeeaf-0101-454a-86a0-da10b89cba84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb7438b6f91542c79eb4067ff42dde07', 'request_id': 'req-c5dfeeaf-0101-454a-86a0-da10b89cba84'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:49.260214 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c5dfeeaf-0101-454a-86a0-da10b89cba84 admin admin] The token cache is being invalidate because project eb7438b6f91542c79eb4067ff42dde07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:49.261261 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c5dfeeaf-0101-454a-86a0-da10b89cba84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:49.262516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c5dfeeaf-0101-454a-86a0-da10b89cba84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:49.264774 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1077/2126] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:49 2022] DELETE /identity/v3/projects/eb7438b6f91542c79eb4067ff42dde07 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:49.268227 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4aaf5819-64af-488c-bb0f-d4de3bd0ac10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb7e4dce11704c83a5dc447245fb8118', 'request_id': 'req-4aaf5819-64af-488c-bb0f-d4de3bd0ac10'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:49.268894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfeeaf-0101-454a-86a0-da10b89cba84 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:49.270449 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c5dfeeaf-0101-454a-86a0-da10b89cba84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:49.278646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4aaf5819-64af-488c-bb0f-d4de3bd0ac10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb7e4dce11704c83a5dc447245fb8118', 'request_id': 'req-4aaf5819-64af-488c-bb0f-d4de3bd0ac10'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:49.286990 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4aaf5819-64af-488c-bb0f-d4de3bd0ac10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb7e4dce11704c83a5dc447245fb8118', 'request_id': 'req-4aaf5819-64af-488c-bb0f-d4de3bd0ac10'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:49.298360 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4aaf5819-64af-488c-bb0f-d4de3bd0ac10 admin admin] The token cache is being invalidate because project cb7e4dce11704c83a5dc447245fb8118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:49.298659 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4aaf5819-64af-488c-bb0f-d4de3bd0ac10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:49.299689 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4aaf5819-64af-488c-bb0f-d4de3bd0ac10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:49.305439 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1050/2127] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:49 2022] DELETE /identity/v3/projects/cb7e4dce11704c83a5dc447245fb8118 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:49.306931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021f5da-7825-4107-8d87-4d2f6893fb09 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:49.307221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021f5da-7825-4107-8d87-4d2f6893fb09 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:49.307502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021f5da-7825-4107-8d87-4d2f6893fb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:49.326464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-4021f5da-7825-4107-8d87-4d2f6893fb09 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:49.327695 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4021f5da-7825-4107-8d87-4d2f6893fb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:49.360440 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfeeaf-0101-454a-86a0-da10b89cba84 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:49.393465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1051/2128] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:49.401628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf2943-8105-4bad-8001-49c8f1b8299d None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:49.401919 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf2943-8105-4bad-8001-49c8f1b8299d None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:49.402190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf2943-8105-4bad-8001-49c8f1b8299d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:49.422588 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae4b44-d702-4d04-a922-b29ce7dbec6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:49.423392 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-6cbf2943-8105-4bad-8001-49c8f1b8299d None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:49.423510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae4b44-d702-4d04-a922-b29ce7dbec6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:49.423747 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae4b44-d702-4d04-a922-b29ce7dbec6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:49.424090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae4b44-d702-4d04-a922-b29ce7dbec6f admin admin] PATH_INFO: `/v3/users/4f40d26bb40b4b9e9804ec3d4cffce0d` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:49.424676 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6cbf2943-8105-4bad-8001-49c8f1b8299d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:49.425346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae4b44-d702-4d04-a922-b29ce7dbec6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f40d26bb40b4b9e9804ec3d4cffce0d)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:49.427273 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae4b44-d702-4d04-a922-b29ce7dbec6f admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:49.434619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1052/2129] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:49.439009 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf2943-8105-4bad-8001-49c8f1b8299d None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:49.449708 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf2943-8105-4bad-8001-49c8f1b8299d None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:49.456732 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23ff13-3173-4788-bf7a-d91efa2b7e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:49.457610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23ff13-3173-4788-bf7a-d91efa2b7e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:49.457907 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23ff13-3173-4788-bf7a-d91efa2b7e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:49.458250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23ff13-3173-4788-bf7a-d91efa2b7e18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:49.459013 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23ff13-3173-4788-bf7a-d91efa2b7e18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:49.459326 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23ff13-3173-4788-bf7a-d91efa2b7e18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:49.461312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23ff13-3173-4788-bf7a-d91efa2b7e18 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:49.481662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1053/2130] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 22:06:49.482680 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23ff13-3173-4788-bf7a-d91efa2b7e18 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:49.493218 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23ff13-3173-4788-bf7a-d91efa2b7e18 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:49.495049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-deae4b44-d702-4d04-a922-b29ce7dbec6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f40d26bb40b4b9e9804ec3d4cffce0d'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:49.497902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfca74-6ad6-4c66-b06d-8a1dc0c65628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:49.498554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfca74-6ad6-4c66-b06d-8a1dc0c65628 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:49.498843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfca74-6ad6-4c66-b06d-8a1dc0c65628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:49.499129 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfca74-6ad6-4c66-b06d-8a1dc0c65628 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:49.499815 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cfca74-6ad6-4c66-b06d-8a1dc0c65628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:49.501798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cfca74-6ad6-4c66-b06d-8a1dc0c65628 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:49.507573 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-deae4b44-d702-4d04-a922-b29ce7dbec6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f40d26bb40b4b9e9804ec3d4cffce0d'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:49.515042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-deae4b44-d702-4d04-a922-b29ce7dbec6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f40d26bb40b4b9e9804ec3d4cffce0d'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:49.522770 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-deae4b44-d702-4d04-a922-b29ce7dbec6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f40d26bb40b4b9e9804ec3d4cffce0d'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:49.523411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-deae4b44-d702-4d04-a922-b29ce7dbec6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f40d26bb40b4b9e9804ec3d4cffce0d'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:49.528431 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1054/2131] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:49 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:49.532748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1078/2132] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:49 2022] DELETE /identity/v3/users/4f40d26bb40b4b9e9804ec3d4cffce0d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:49.533853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfca74-6ad6-4c66-b06d-8a1dc0c65628 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:49.535561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-68cfca74-6ad6-4c66-b06d-8a1dc0c65628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:49.542158 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:49.629250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfca74-6ad6-4c66-b06d-8a1dc0c65628 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:49.710594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e39fda-ac1e-4e05-a43b-bba8b2b737f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:49.711242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e39fda-ac1e-4e05-a43b-bba8b2b737f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:49.711940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e39fda-ac1e-4e05-a43b-bba8b2b737f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:49.711940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e39fda-ac1e-4e05-a43b-bba8b2b737f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:49.712479 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e39fda-ac1e-4e05-a43b-bba8b2b737f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:49.714458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e39fda-ac1e-4e05-a43b-bba8b2b737f7 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:49.742487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1055/2133] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:49 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:06:49.746647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae4b44-d702-4d04-a922-b29ce7dbec6f admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:49.759501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae4b44-d702-4d04-a922-b29ce7dbec6f admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:49.765795 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc34357-4733-436b-96f4-76c07b5a1391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:49.766701 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc34357-4733-436b-96f4-76c07b5a1391 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:49.766988 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc34357-4733-436b-96f4-76c07b5a1391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:49.767475 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc34357-4733-436b-96f4-76c07b5a1391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:49.768012 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc34357-4733-436b-96f4-76c07b5a1391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:49.769666 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc34357-4733-436b-96f4-76c07b5a1391 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:49.774749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:49.788845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1079/2134] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:49.793450 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e39fda-ac1e-4e05-a43b-bba8b2b737f7 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:49.805364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e39fda-ac1e-4e05-a43b-bba8b2b737f7 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:49.810425 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba288946-2fcb-409e-b41d-86729947baf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:49.811139 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba288946-2fcb-409e-b41d-86729947baf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:49.811434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba288946-2fcb-409e-b41d-86729947baf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:49.811713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba288946-2fcb-409e-b41d-86729947baf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:49.812446 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba288946-2fcb-409e-b41d-86729947baf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:49.814096 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba288946-2fcb-409e-b41d-86729947baf1 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:49.829035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1056/2135] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:49.834179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc34357-4733-436b-96f4-76c07b5a1391 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:49.846627 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc34357-4733-436b-96f4-76c07b5a1391 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:49.852688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80187497-a0cd-48db-89c6-a13c2b4f37f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:49.854644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80187497-a0cd-48db-89c6-a13c2b4f37f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:49.854936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80187497-a0cd-48db-89c6-a13c2b4f37f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:49.855218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80187497-a0cd-48db-89c6-a13c2b4f37f5 admin admin] PATH_INFO: `/v3/projects/304ae4bdbbf74ec9896a31d1f72ed72a/users/64f97c33dcc74dd4ad69c277415f4643/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:49.862364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba288946-2fcb-409e-b41d-86729947baf1 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:49.864854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ba288946-2fcb-409e-b41d-86729947baf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:49.876970 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80187497-a0cd-48db-89c6-a13c2b4f37f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=304ae4bdbbf74ec9896a31d1f72ed72a, user_id=64f97c33dcc74dd4ad69c277415f4643, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:49.882186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80187497-a0cd-48db-89c6-a13c2b4f37f5 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:49.891332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba288946-2fcb-409e-b41d-86729947baf1 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:49.902212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1080/2136] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:49 2022] PUT /identity/v3/projects/304ae4bdbbf74ec9896a31d1f72ed72a/users/64f97c33dcc74dd4ad69c277415f4643/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:49.910540 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826281f-8109-4aba-b1e7-7615681e646c None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:49.910857 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826281f-8109-4aba-b1e7-7615681e646c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:49.911146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826281f-8109-4aba-b1e7-7615681e646c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:49.932445 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-d826281f-8109-4aba-b1e7-7615681e646c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:49.933732 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d826281f-8109-4aba-b1e7-7615681e646c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:49.978523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da43bae-1ce8-41ed-b405-95253c7678a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:49.979778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da43bae-1ce8-41ed-b405-95253c7678a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:49.980276 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da43bae-1ce8-41ed-b405-95253c7678a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:49.980548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da43bae-1ce8-41ed-b405-95253c7678a5 admin admin] PATH_INFO: `/v3/projects/79be3adf1c9a43a8bc0bb9b132c80f15` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:49.982097 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da43bae-1ce8-41ed-b405-95253c7678a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79be3adf1c9a43a8bc0bb9b132c80f15)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:49.984228 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da43bae-1ce8-41ed-b405-95253c7678a5 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:50.032777 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1081/2137] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:50.041388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:50.049215 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826281f-8109-4aba-b1e7-7615681e646c None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:50.056835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d826281f-8109-4aba-b1e7-7615681e646c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:50.057451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6da43bae-1ce8-41ed-b405-95253c7678a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79be3adf1c9a43a8bc0bb9b132c80f15', 'request_id': 'req-6da43bae-1ce8-41ed-b405-95253c7678a5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:50.065760 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6da43bae-1ce8-41ed-b405-95253c7678a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79be3adf1c9a43a8bc0bb9b132c80f15', 'request_id': 'req-6da43bae-1ce8-41ed-b405-95253c7678a5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:50.077801 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6da43bae-1ce8-41ed-b405-95253c7678a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79be3adf1c9a43a8bc0bb9b132c80f15', 'request_id': 'req-6da43bae-1ce8-41ed-b405-95253c7678a5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:50.098889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6da43bae-1ce8-41ed-b405-95253c7678a5 admin admin] The token cache is being invalidate because project 79be3adf1c9a43a8bc0bb9b132c80f15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:50.099335 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6da43bae-1ce8-41ed-b405-95253c7678a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:50.099981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6da43bae-1ce8-41ed-b405-95253c7678a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:50.102159 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1057/2138] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:49 2022] DELETE /identity/v3/projects/79be3adf1c9a43a8bc0bb9b132c80f15 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:50.109852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039d4d5-87ee-4ed5-b1d7-03becf427604 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:50.110141 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039d4d5-87ee-4ed5-b1d7-03becf427604 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:50.110461 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039d4d5-87ee-4ed5-b1d7-03becf427604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:50.137856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-9039d4d5-87ee-4ed5-b1d7-03becf427604 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:50.140743 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-9039d4d5-87ee-4ed5-b1d7-03becf427604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:50.155234 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826281f-8109-4aba-b1e7-7615681e646c None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:50.258105 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1058/2139] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:50.267084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89dfc2-9e55-4c18-a4a1-66605a9a18d1 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:50.267443 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89dfc2-9e55-4c18-a4a1-66605a9a18d1 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:50.267672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89dfc2-9e55-4c18-a4a1-66605a9a18d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:50.269870 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461555c-6656-420c-a86c-177844cfff14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:50.270901 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461555c-6656-420c-a86c-177844cfff14 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:50.271257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461555c-6656-420c-a86c-177844cfff14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:50.271600 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461555c-6656-420c-a86c-177844cfff14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:50.275167 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0461555c-6656-420c-a86c-177844cfff14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:50.292854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-9b89dfc2-9e55-4c18-a4a1-66605a9a18d1 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:50.294224 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-9b89dfc2-9e55-4c18-a4a1-66605a9a18d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:50.296264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461555c-6656-420c-a86c-177844cfff14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:50.298087 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461555c-6656-420c-a86c-177844cfff14 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:50.307380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1059/2140] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:50.309975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89dfc2-9e55-4c18-a4a1-66605a9a18d1 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:50.311952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1082/2141] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:50.321989 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89dfc2-9e55-4c18-a4a1-66605a9a18d1 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:50.328217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d1232-7fc0-4fda-a393-c25de1842a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:50.329263 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d1232-7fc0-4fda-a393-c25de1842a63 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:50.330123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d1232-7fc0-4fda-a393-c25de1842a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:50.330443 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d1232-7fc0-4fda-a393-c25de1842a63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:50.331195 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d1232-7fc0-4fda-a393-c25de1842a63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:06:50.331679 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d1232-7fc0-4fda-a393-c25de1842a63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:50.333362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d1232-7fc0-4fda-a393-c25de1842a63 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:50.355114 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1060/2142] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 22:06:50.361119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461555c-6656-420c-a86c-177844cfff14 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:50.383100 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461555c-6656-420c-a86c-177844cfff14 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:50.388255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf9651-a48b-4f5e-8f2c-6d7443568b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:50.388992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf9651-a48b-4f5e-8f2c-6d7443568b2a admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:50.389284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf9651-a48b-4f5e-8f2c-6d7443568b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:50.389566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf9651-a48b-4f5e-8f2c-6d7443568b2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:50.390264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf9651-a48b-4f5e-8f2c-6d7443568b2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:50.392211 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf9651-a48b-4f5e-8f2c-6d7443568b2a admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:50.441625 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1083/2143] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:06:50 2022] POST /identity/v3/projects => generated 385 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:06:50.447463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d1232-7fc0-4fda-a393-c25de1842a63 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:50.468540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d1232-7fc0-4fda-a393-c25de1842a63 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:50.546992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87315ac-882d-4d84-8052-87e7de73acd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:50.547684 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87315ac-882d-4d84-8052-87e7de73acd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:50.548025 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87315ac-882d-4d84-8052-87e7de73acd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:50.548438 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87315ac-882d-4d84-8052-87e7de73acd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:50.549312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87315ac-882d-4d84-8052-87e7de73acd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:50.551245 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87315ac-882d-4d84-8052-87e7de73acd5 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:50.577182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1061/2144] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:06:50 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:06:50.578372 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf9651-a48b-4f5e-8f2c-6d7443568b2a admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:50.589560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf9651-a48b-4f5e-8f2c-6d7443568b2a admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:50.594347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76909fb8-91ae-4a78-83d1-d20733bb30f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:50.595011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76909fb8-91ae-4a78-83d1-d20733bb30f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:50.595300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76909fb8-91ae-4a78-83d1-d20733bb30f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:50.595578 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76909fb8-91ae-4a78-83d1-d20733bb30f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:50.596308 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76909fb8-91ae-4a78-83d1-d20733bb30f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:50.597956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76909fb8-91ae-4a78-83d1-d20733bb30f0 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:50.604880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:50.612595 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1084/2145] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:50.616938 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87315ac-882d-4d84-8052-87e7de73acd5 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:50.627903 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87315ac-882d-4d84-8052-87e7de73acd5 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:50.633668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe171e2-91b0-4a4f-805a-30065d57d15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:50.634341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe171e2-91b0-4a4f-805a-30065d57d15e admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:50.634628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe171e2-91b0-4a4f-805a-30065d57d15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:50.634907 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe171e2-91b0-4a4f-805a-30065d57d15e admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:50.635641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe171e2-91b0-4a4f-805a-30065d57d15e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:50.637605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe171e2-91b0-4a4f-805a-30065d57d15e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:50.650980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1062/2146] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:06:50.656890 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76909fb8-91ae-4a78-83d1-d20733bb30f0 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:50.677810 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76909fb8-91ae-4a78-83d1-d20733bb30f0 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:50.683763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce7215-539d-4671-80a6-76bd31d2e18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:50.684644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce7215-539d-4671-80a6-76bd31d2e18f admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:50.684961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce7215-539d-4671-80a6-76bd31d2e18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:50.685275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce7215-539d-4671-80a6-76bd31d2e18f admin admin] PATH_INFO: `/v3/projects/bb2f3ef394174aababcf56eacf6ff0eb/users/7cbe673ad1b0434a8faedf2b86680336/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:50.706542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce7215-539d-4671-80a6-76bd31d2e18f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb2f3ef394174aababcf56eacf6ff0eb, user_id=7cbe673ad1b0434a8faedf2b86680336, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:50.709447 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce7215-539d-4671-80a6-76bd31d2e18f admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:50.732261 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1085/2147] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:06:50 2022] PUT /identity/v3/projects/bb2f3ef394174aababcf56eacf6ff0eb/users/7cbe673ad1b0434a8faedf2b86680336/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:06:50.743332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd3720-f43f-4089-92f6-82deeca5feb7 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:50.743621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd3720-f43f-4089-92f6-82deeca5feb7 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:50.743901 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd3720-f43f-4089-92f6-82deeca5feb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:50.777736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-bdcd3720-f43f-4089-92f6-82deeca5feb7 None None] MFA Rules not processed for user `7cbe673ad1b0434a8faedf2b86680336`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:50.778928 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-bdcd3720-f43f-4089-92f6-82deeca5feb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:50.791067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:50.876588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1063/2148] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:50.894855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce7215-539d-4671-80a6-76bd31d2e18f admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:50.898416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e0ce7215-539d-4671-80a6-76bd31d2e18f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:50.985572 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce7215-539d-4671-80a6-76bd31d2e18f admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:50.991209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efcd4d-842a-4461-a897-859a0b25bfad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:50.991913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efcd4d-842a-4461-a897-859a0b25bfad service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:50.992233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efcd4d-842a-4461-a897-859a0b25bfad service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:50.992601 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efcd4d-842a-4461-a897-859a0b25bfad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:51.001715 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efcd4d-842a-4461-a897-859a0b25bfad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:51.003154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efcd4d-842a-4461-a897-859a0b25bfad service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:51.013921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1086/2149] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:51.053650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22da27b-c431-4fed-aa50-aa154fbe8b9d None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:51.053650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22da27b-c431-4fed-aa50-aa154fbe8b9d None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:51.053650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22da27b-c431-4fed-aa50-aa154fbe8b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:51.094162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-f22da27b-c431-4fed-aa50-aa154fbe8b9d None None] MFA Rules not processed for user `64f97c33dcc74dd4ad69c277415f4643`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:51.095921 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-f22da27b-c431-4fed-aa50-aa154fbe8b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:51.215673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1064/2150] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:51.258169 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efcd4d-842a-4461-a897-859a0b25bfad service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:51.259860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-79efcd4d-842a-4461-a897-859a0b25bfad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:51.274475 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efcd4d-842a-4461-a897-859a0b25bfad service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:51.278495 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc95aea-486b-499b-84b9-e56d1888baf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:51.279474 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc95aea-486b-499b-84b9-e56d1888baf3 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:51.279764 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc95aea-486b-499b-84b9-e56d1888baf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:51.280086 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc95aea-486b-499b-84b9-e56d1888baf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:51.294097 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc95aea-486b-499b-84b9-e56d1888baf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:51.295533 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc95aea-486b-499b-84b9-e56d1888baf3 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:51.305448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1087/2151] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:51.403782 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22da27b-c431-4fed-aa50-aa154fbe8b9d None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:51.414826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22da27b-c431-4fed-aa50-aa154fbe8b9d None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:51.424621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d7e32-308e-4141-8fd7-174f5cc4e6e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:51.427879 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d7e32-308e-4141-8fd7-174f5cc4e6e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:51.428973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d7e32-308e-4141-8fd7-174f5cc4e6e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:51.429326 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d7e32-308e-4141-8fd7-174f5cc4e6e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:51.435829 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ea0d7e32-308e-4141-8fd7-174f5cc4e6e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:51.517755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d7e32-308e-4141-8fd7-174f5cc4e6e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:51.519390 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d7e32-308e-4141-8fd7-174f5cc4e6e3 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:51.530982 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1065/2152] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:52.869418 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518234e-242f-41cd-8c6e-df61ac05435f None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:52.869912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518234e-242f-41cd-8c6e-df61ac05435f None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:52.870436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518234e-242f-41cd-8c6e-df61ac05435f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:52.889908 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-c518234e-242f-41cd-8c6e-df61ac05435f None None] MFA Rules not processed for user `64f97c33dcc74dd4ad69c277415f4643`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:52.891151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c518234e-242f-41cd-8c6e-df61ac05435f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:52.899162 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1088/2153] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:54.005522 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d7e32-308e-4141-8fd7-174f5cc4e6e3 service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:54.057110 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:54.102677 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d7e32-308e-4141-8fd7-174f5cc4e6e3 service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:54.107952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ad55d-0151-4f96-a597-7d7a127e4ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:54.109071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ad55d-0151-4f96-a597-7d7a127e4ae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:54.109507 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ad55d-0151-4f96-a597-7d7a127e4ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:54.109779 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ad55d-0151-4f96-a597-7d7a127e4ae6 admin admin] PATH_INFO: `/v3/users/7cbe673ad1b0434a8faedf2b86680336` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:54.111124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ad55d-0151-4f96-a597-7d7a127e4ae6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cbe673ad1b0434a8faedf2b86680336)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:54.113109 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ad55d-0151-4f96-a597-7d7a127e4ae6 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:54.168725 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8a6ad55d-0151-4f96-a597-7d7a127e4ae6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cbe673ad1b0434a8faedf2b86680336'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:54.178037 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8a6ad55d-0151-4f96-a597-7d7a127e4ae6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cbe673ad1b0434a8faedf2b86680336'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:54.190077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8a6ad55d-0151-4f96-a597-7d7a127e4ae6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cbe673ad1b0434a8faedf2b86680336'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:54.190778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8a6ad55d-0151-4f96-a597-7d7a127e4ae6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cbe673ad1b0434a8faedf2b86680336'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:54.191535 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8a6ad55d-0151-4f96-a597-7d7a127e4ae6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cbe673ad1b0434a8faedf2b86680336'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:54.213733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1066/2154] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:54 2022] DELETE /identity/v3/users/7cbe673ad1b0434a8faedf2b86680336 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:54.218387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb11b0-d383-4e07-aea4-9db79927309c None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:54.218844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb11b0-d383-4e07-aea4-9db79927309c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:54.218959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb11b0-d383-4e07-aea4-9db79927309c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:54.241300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-a5eb11b0-d383-4e07-aea4-9db79927309c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:54.242516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a5eb11b0-d383-4e07-aea4-9db79927309c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:54.307779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1089/2155] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:06:54.318267 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ad55d-0151-4f96-a597-7d7a127e4ae6 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:54.320036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8a6ad55d-0151-4f96-a597-7d7a127e4ae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:54.384222 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ad55d-0151-4f96-a597-7d7a127e4ae6 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:54.388576 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d51eb-8d8d-418e-87df-dbb461867028 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:54.389274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d51eb-8d8d-418e-87df-dbb461867028 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:54.389555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d51eb-8d8d-418e-87df-dbb461867028 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:54.389836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d51eb-8d8d-418e-87df-dbb461867028 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:54.402434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d51eb-8d8d-418e-87df-dbb461867028 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:54.404305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d51eb-8d8d-418e-87df-dbb461867028 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:54.417065 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1067/2156] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:54.830282 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb11b0-d383-4e07-aea4-9db79927309c None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:54.831955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a5eb11b0-d383-4e07-aea4-9db79927309c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:54.848023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb11b0-d383-4e07-aea4-9db79927309c None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:54.855456 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12e5a6-91bc-4c77-a098-ece5023acc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:54.856929 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12e5a6-91bc-4c77-a098-ece5023acc9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:54.857441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12e5a6-91bc-4c77-a098-ece5023acc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:54.857787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12e5a6-91bc-4c77-a098-ece5023acc9d admin admin] PATH_INFO: `/v3/projects/bb2f3ef394174aababcf56eacf6ff0eb` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:54.859118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12e5a6-91bc-4c77-a098-ece5023acc9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb2f3ef394174aababcf56eacf6ff0eb)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:54.861218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12e5a6-91bc-4c77-a098-ece5023acc9d admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:54.931676 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2d12e5a6-91bc-4c77-a098-ece5023acc9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb2f3ef394174aababcf56eacf6ff0eb', 'request_id': 'req-2d12e5a6-91bc-4c77-a098-ece5023acc9d'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:54.936054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:54.938347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2d12e5a6-91bc-4c77-a098-ece5023acc9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb2f3ef394174aababcf56eacf6ff0eb', 'request_id': 'req-2d12e5a6-91bc-4c77-a098-ece5023acc9d'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:54.948304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2d12e5a6-91bc-4c77-a098-ece5023acc9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb2f3ef394174aababcf56eacf6ff0eb', 'request_id': 'req-2d12e5a6-91bc-4c77-a098-ece5023acc9d'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:54.970625 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2d12e5a6-91bc-4c77-a098-ece5023acc9d admin admin] The token cache is being invalidate because project bb2f3ef394174aababcf56eacf6ff0eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:06:54.971159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2d12e5a6-91bc-4c77-a098-ece5023acc9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:54.972453 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2d12e5a6-91bc-4c77-a098-ece5023acc9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:54.974174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1090/2157] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:54 2022] DELETE /identity/v3/projects/bb2f3ef394174aababcf56eacf6ff0eb => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:58.752519 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d51eb-8d8d-418e-87df-dbb461867028 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:58.753986 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-124d51eb-8d8d-418e-87df-dbb461867028 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:58.777168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d51eb-8d8d-418e-87df-dbb461867028 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:58.876102 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:06:58.901960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a82b0-8763-4a0b-bd42-3548068bdec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:58.902607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a82b0-8763-4a0b-bd42-3548068bdec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:58.902953 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a82b0-8763-4a0b-bd42-3548068bdec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:58.903298 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a82b0-8763-4a0b-bd42-3548068bdec5 admin admin] PATH_INFO: `/v3/users/238236d7b4284966b8989c6dea082821` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:58.904625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a82b0-8763-4a0b-bd42-3548068bdec5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=238236d7b4284966b8989c6dea082821)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:58.906672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a82b0-8763-4a0b-bd42-3548068bdec5 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:58.989158 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-be7a82b0-8763-4a0b-bd42-3548068bdec5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '238236d7b4284966b8989c6dea082821'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:59.001142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-be7a82b0-8763-4a0b-bd42-3548068bdec5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '238236d7b4284966b8989c6dea082821'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:59.011725 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-be7a82b0-8763-4a0b-bd42-3548068bdec5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '238236d7b4284966b8989c6dea082821'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:59.012468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-be7a82b0-8763-4a0b-bd42-3548068bdec5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '238236d7b4284966b8989c6dea082821'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:59.013262 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-be7a82b0-8763-4a0b-bd42-3548068bdec5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '238236d7b4284966b8989c6dea082821'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:59.040473 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1068/2158] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:58 2022] DELETE /identity/v3/users/238236d7b4284966b8989c6dea082821 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:59.045155 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7474a-dbfc-4fcc-893c-5f07b17dc66c None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:59.045155 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7474a-dbfc-4fcc-893c-5f07b17dc66c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:59.045752 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7474a-dbfc-4fcc-893c-5f07b17dc66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:59.065062 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-79c7474a-dbfc-4fcc-893c-5f07b17dc66c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:06:59.066479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-79c7474a-dbfc-4fcc-893c-5f07b17dc66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:59.174533 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1091/2159] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:06:59.191464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a82b0-8763-4a0b-bd42-3548068bdec5 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:59.192939 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-be7a82b0-8763-4a0b-bd42-3548068bdec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:59.302175 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a82b0-8763-4a0b-bd42-3548068bdec5 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:59.308645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7bff4-3a01-4199-a4ba-d265573fd015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:59.309463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7bff4-3a01-4199-a4ba-d265573fd015 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:59.309846 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7bff4-3a01-4199-a4ba-d265573fd015 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:59.312253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7bff4-3a01-4199-a4ba-d265573fd015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:59.327274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7bff4-3a01-4199-a4ba-d265573fd015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:59.329889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7bff4-3a01-4199-a4ba-d265573fd015 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:59.347406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1069/2160] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:06:59.705425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7474a-dbfc-4fcc-893c-5f07b17dc66c None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:59.706859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-79c7474a-dbfc-4fcc-893c-5f07b17dc66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:59.719890 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7474a-dbfc-4fcc-893c-5f07b17dc66c None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:06:59.725928 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095df829-1dca-4075-810f-fb3c2c57e9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:06:59.726979 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095df829-1dca-4075-810f-fb3c2c57e9c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:06:59.727264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095df829-1dca-4075-810f-fb3c2c57e9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:06:59.727539 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095df829-1dca-4075-810f-fb3c2c57e9c1 admin admin] PATH_INFO: `/v3/users/bf13eeff770b406796ab7008b8429637` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:06:59.728918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095df829-1dca-4075-810f-fb3c2c57e9c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf13eeff770b406796ab7008b8429637)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:06:59.730853 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095df829-1dca-4075-810f-fb3c2c57e9c1 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:06:59.802837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-095df829-1dca-4075-810f-fb3c2c57e9c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf13eeff770b406796ab7008b8429637'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:59.816741 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-095df829-1dca-4075-810f-fb3c2c57e9c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf13eeff770b406796ab7008b8429637'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:59.824566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-095df829-1dca-4075-810f-fb3c2c57e9c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf13eeff770b406796ab7008b8429637'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:59.835201 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-095df829-1dca-4075-810f-fb3c2c57e9c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf13eeff770b406796ab7008b8429637'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:59.835885 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-095df829-1dca-4075-810f-fb3c2c57e9c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf13eeff770b406796ab7008b8429637'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:06:59.843200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1092/2161] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:06:59 2022] DELETE /identity/v3/users/bf13eeff770b406796ab7008b8429637 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:06:59.945432 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7bff4-3a01-4199-a4ba-d265573fd015 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:06:59.947517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1cd7bff4-3a01-4199-a4ba-d265573fd015 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:06:59.977440 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7bff4-3a01-4199-a4ba-d265573fd015 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:00.045174 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:00.046113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:00.047061 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:00.047348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41 admin admin] PATH_INFO: `/v3/projects/ebecae0398ee4e99a6b7ef9260bc39b3` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:00.048869 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebecae0398ee4e99a6b7ef9260bc39b3)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:00.050917 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:00.074897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:00.098972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095df829-1dca-4075-810f-fb3c2c57e9c1 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:00.101337 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-095df829-1dca-4075-810f-fb3c2c57e9c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:00.111642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebecae0398ee4e99a6b7ef9260bc39b3', 'request_id': 'req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.118662 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095df829-1dca-4075-810f-fb3c2c57e9c1 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:00.120119 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebecae0398ee4e99a6b7ef9260bc39b3', 'request_id': 'req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.129266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebecae0398ee4e99a6b7ef9260bc39b3', 'request_id': 'req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.147793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41 admin admin] The token cache is being invalidate because project ebecae0398ee4e99a6b7ef9260bc39b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:07:00.148477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.149370 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.151694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1070/2162] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:06:59 2022] DELETE /identity/v3/projects/ebecae0398ee4e99a6b7ef9260bc39b3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:00.155533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:00.156940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:00.177697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:00.186812 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be652cc9-ed02-44e2-9aa1-7854981e821a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:00.187706 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be652cc9-ed02-44e2-9aa1-7854981e821a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:00.188153 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be652cc9-ed02-44e2-9aa1-7854981e821a admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:00.188432 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be652cc9-ed02-44e2-9aa1-7854981e821a admin admin] PATH_INFO: `/v3/projects/bdc783af528d4cd2a2b05519449924d9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:00.190182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be652cc9-ed02-44e2-9aa1-7854981e821a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc783af528d4cd2a2b05519449924d9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:00.192415 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be652cc9-ed02-44e2-9aa1-7854981e821a admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:00.227270 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b728bc-fa81-4a50-b1c6-099a2bdf4f41 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:00.232367 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6ea18-c707-4e0b-92ab-b0c521f3e7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:00.233328 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6ea18-c707-4e0b-92ab-b0c521f3e7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:00.233669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6ea18-c707-4e0b-92ab-b0c521f3e7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:00.233974 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6ea18-c707-4e0b-92ab-b0c521f3e7c6 admin admin] PATH_INFO: `/v3/users/19e4161ee978401995608fc53a13f4d3` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:00.235215 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6ea18-c707-4e0b-92ab-b0c521f3e7c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19e4161ee978401995608fc53a13f4d3)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:00.237291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6ea18-c707-4e0b-92ab-b0c521f3e7c6 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:00.246558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-be652cc9-ed02-44e2-9aa1-7854981e821a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc783af528d4cd2a2b05519449924d9', 'request_id': 'req-be652cc9-ed02-44e2-9aa1-7854981e821a'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.252455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-be652cc9-ed02-44e2-9aa1-7854981e821a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc783af528d4cd2a2b05519449924d9', 'request_id': 'req-be652cc9-ed02-44e2-9aa1-7854981e821a'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.261796 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-be652cc9-ed02-44e2-9aa1-7854981e821a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc783af528d4cd2a2b05519449924d9', 'request_id': 'req-be652cc9-ed02-44e2-9aa1-7854981e821a'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.274778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-be652cc9-ed02-44e2-9aa1-7854981e821a admin admin] The token cache is being invalidate because project bdc783af528d4cd2a2b05519449924d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:07:00.275133 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-be652cc9-ed02-44e2-9aa1-7854981e821a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.276045 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-be652cc9-ed02-44e2-9aa1-7854981e821a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.278171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1093/2163] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:07:00 2022] DELETE /identity/v3/projects/bdc783af528d4cd2a2b05519449924d9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:07:00.283458 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f326ca7-c473-4348-b803-4788382e8e3f None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:00.283907 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f326ca7-c473-4348-b803-4788382e8e3f None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:00.284224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f326ca7-c473-4348-b803-4788382e8e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:00.293566 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-34e6ea18-c707-4e0b-92ab-b0c521f3e7c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19e4161ee978401995608fc53a13f4d3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.301001 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-34e6ea18-c707-4e0b-92ab-b0c521f3e7c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19e4161ee978401995608fc53a13f4d3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.303986 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-4f326ca7-c473-4348-b803-4788382e8e3f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:00.305443 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-4f326ca7-c473-4348-b803-4788382e8e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:00.309806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-34e6ea18-c707-4e0b-92ab-b0c521f3e7c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19e4161ee978401995608fc53a13f4d3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.310441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-34e6ea18-c707-4e0b-92ab-b0c521f3e7c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19e4161ee978401995608fc53a13f4d3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.311189 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-34e6ea18-c707-4e0b-92ab-b0c521f3e7c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19e4161ee978401995608fc53a13f4d3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.336038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1071/2164] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:07:00 2022] DELETE /identity/v3/users/19e4161ee978401995608fc53a13f4d3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:07:00.374648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1094/2165] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:07:00.383171 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2df41-3feb-4cd4-8c71-652f40668e67 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:00.383520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2df41-3feb-4cd4-8c71-652f40668e67 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:00.383688 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2df41-3feb-4cd4-8c71-652f40668e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:00.406115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-aad2df41-3feb-4cd4-8c71-652f40668e67 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:00.407324 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-aad2df41-3feb-4cd4-8c71-652f40668e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:00.431186 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:00.478122 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1072/2166] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:00.482873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f326ca7-c473-4348-b803-4788382e8e3f None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:00.494589 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f326ca7-c473-4348-b803-4788382e8e3f None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:00.502085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962771f-c6b7-4d56-9c49-cdb952cb3c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:00.502895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962771f-c6b7-4d56-9c49-cdb952cb3c56 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:00.503238 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962771f-c6b7-4d56-9c49-cdb952cb3c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:00.503585 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962771f-c6b7-4d56-9c49-cdb952cb3c56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:00.504523 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962771f-c6b7-4d56-9c49-cdb952cb3c56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:07:00.505219 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962771f-c6b7-4d56-9c49-cdb952cb3c56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:00.507199 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962771f-c6b7-4d56-9c49-cdb952cb3c56 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:00.524767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1095/2167] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:07:00.527626 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2df41-3feb-4cd4-8c71-652f40668e67 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:00.538927 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2df41-3feb-4cd4-8c71-652f40668e67 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:00.544462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3beb59-2f2d-4bc2-bc47-627c8780baef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:00.545181 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3beb59-2f2d-4bc2-bc47-627c8780baef admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:00.545483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3beb59-2f2d-4bc2-bc47-627c8780baef admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:00.545758 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3beb59-2f2d-4bc2-bc47-627c8780baef admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:00.546565 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3beb59-2f2d-4bc2-bc47-627c8780baef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:00.550518 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3beb59-2f2d-4bc2-bc47-627c8780baef admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:00.592243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1073/2168] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:07:00 2022] POST /identity/v3/projects => generated 393 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:07:00.596556 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962771f-c6b7-4d56-9c49-cdb952cb3c56 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:00.607613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962771f-c6b7-4d56-9c49-cdb952cb3c56 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:00.671295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a8eb5-59f7-4306-866e-bc214808a272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:00.672363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a8eb5-59f7-4306-866e-bc214808a272 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:00.672832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a8eb5-59f7-4306-866e-bc214808a272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:00.673267 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a8eb5-59f7-4306-866e-bc214808a272 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:00.674477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a8eb5-59f7-4306-866e-bc214808a272 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:00.676984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a8eb5-59f7-4306-866e-bc214808a272 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:00.695673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3beb59-2f2d-4bc2-bc47-627c8780baef admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:00.697665 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-2e3beb59-2f2d-4bc2-bc47-627c8780baef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:00.707839 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1096/2169] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:07:00 2022] POST /identity/v3/users => generated 297 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:07:00.713134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a8eb5-59f7-4306-866e-bc214808a272 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:00.720666 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:00.727930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3beb59-2f2d-4bc2-bc47-627c8780baef admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:00.729877 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a8eb5-59f7-4306-866e-bc214808a272 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:00.746273 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f8d1b-2145-41e1-ba62-76cf7fbc5137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:00.747069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f8d1b-2145-41e1-ba62-76cf7fbc5137 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:00.747346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f8d1b-2145-41e1-ba62-76cf7fbc5137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:00.747636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f8d1b-2145-41e1-ba62-76cf7fbc5137 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:00.748379 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f8d1b-2145-41e1-ba62-76cf7fbc5137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:00.750120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f8d1b-2145-41e1-ba62-76cf7fbc5137 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:00.751519 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf89797-f877-4d1b-ad05-41f5d8fa2c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:00.752572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf89797-f877-4d1b-ad05-41f5d8fa2c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:00.753565 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf89797-f877-4d1b-ad05-41f5d8fa2c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:00.753959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf89797-f877-4d1b-ad05-41f5d8fa2c03 admin admin] PATH_INFO: `/v3/projects/788b5ea0ea5745bfa44cbee3ec4f3a3e` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:00.755466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf89797-f877-4d1b-ad05-41f5d8fa2c03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=788b5ea0ea5745bfa44cbee3ec4f3a3e)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:00.758175 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf89797-f877-4d1b-ad05-41f5d8fa2c03 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:00.763586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1097/2170] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:07:00.767604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f8d1b-2145-41e1-ba62-76cf7fbc5137 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:00.782507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f8d1b-2145-41e1-ba62-76cf7fbc5137 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:00.787713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba79ed-d007-4bda-9370-96c54492875f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:00.788792 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba79ed-d007-4bda-9370-96c54492875f admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:00.789764 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba79ed-d007-4bda-9370-96c54492875f admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:00.790156 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba79ed-d007-4bda-9370-96c54492875f admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:00.791046 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba79ed-d007-4bda-9370-96c54492875f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:00.793380 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba79ed-d007-4bda-9370-96c54492875f admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:00.824220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1098/2171] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:07:00.825467 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba79ed-d007-4bda-9370-96c54492875f admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:00.836628 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba79ed-d007-4bda-9370-96c54492875f admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:00.846900 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9bf89797-f877-4d1b-ad05-41f5d8fa2c03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '788b5ea0ea5745bfa44cbee3ec4f3a3e', 'request_id': 'req-9bf89797-f877-4d1b-ad05-41f5d8fa2c03'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.856433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9bf89797-f877-4d1b-ad05-41f5d8fa2c03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '788b5ea0ea5745bfa44cbee3ec4f3a3e', 'request_id': 'req-9bf89797-f877-4d1b-ad05-41f5d8fa2c03'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.872947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9bf89797-f877-4d1b-ad05-41f5d8fa2c03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '788b5ea0ea5745bfa44cbee3ec4f3a3e', 'request_id': 'req-9bf89797-f877-4d1b-ad05-41f5d8fa2c03'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.895783 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9bf89797-f877-4d1b-ad05-41f5d8fa2c03 admin admin] The token cache is being invalidate because project 788b5ea0ea5745bfa44cbee3ec4f3a3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:07:00.896072 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9bf89797-f877-4d1b-ad05-41f5d8fa2c03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.898005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9bf89797-f877-4d1b-ad05-41f5d8fa2c03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:00.901175 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1074/2172] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:07:00 2022] DELETE /identity/v3/projects/788b5ea0ea5745bfa44cbee3ec4f3a3e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:00.910029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c93ea-c706-4e2d-984f-2c9d1d6b4129 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:00.911057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c93ea-c706-4e2d-984f-2c9d1d6b4129 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:00.911785 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c93ea-c706-4e2d-984f-2c9d1d6b4129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:00.932043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c097e-7149-4ecb-96ed-7f4d1d8a2bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:00.933183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c097e-7149-4ecb-96ed-7f4d1d8a2bf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:00.933643 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c097e-7149-4ecb-96ed-7f4d1d8a2bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:00.934095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c097e-7149-4ecb-96ed-7f4d1d8a2bf7 admin admin] PATH_INFO: `/v3/projects/4b4f6be5087f4fd18a60969372480425/users/309de655c8d64223a9c2d75ff5618411/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:00.935331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-857c93ea-c706-4e2d-984f-2c9d1d6b4129 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:00.936686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-857c93ea-c706-4e2d-984f-2c9d1d6b4129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:00.948033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1075/2173] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:07:00.957523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59ecb7-1866-4462-bbf2-3632fea1feb3 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:00.957871 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59ecb7-1866-4462-bbf2-3632fea1feb3 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:00.957979 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c097e-7149-4ecb-96ed-7f4d1d8a2bf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b4f6be5087f4fd18a60969372480425, user_id=309de655c8d64223a9c2d75ff5618411, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:00.958151 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59ecb7-1866-4462-bbf2-3632fea1feb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:00.962184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c097e-7149-4ecb-96ed-7f4d1d8a2bf7 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:00.993051 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1099/2174] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:07:00 2022] PUT /identity/v3/projects/4b4f6be5087f4fd18a60969372480425/users/309de655c8d64223a9c2d75ff5618411/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:00.998018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5468e7-a066-434b-a202-a1b20ff6b75b None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:00.998310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5468e7-a066-434b-a202-a1b20ff6b75b None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:00.998597 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5468e7-a066-434b-a202-a1b20ff6b75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:00.999426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-ae59ecb7-1866-4462-bbf2-3632fea1feb3 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:01.000726 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ae59ecb7-1866-4462-bbf2-3632fea1feb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:01.021118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-4e5468e7-a066-434b-a202-a1b20ff6b75b None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:01.023551 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-4e5468e7-a066-434b-a202-a1b20ff6b75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:01.073356 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1076/2175] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:01.077679 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59ecb7-1866-4462-bbf2-3632fea1feb3 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:01.088529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59ecb7-1866-4462-bbf2-3632fea1feb3 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:01.093846 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afeef12-6ac0-43f5-8a27-aceddfe14fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:01.094512 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afeef12-6ac0-43f5-8a27-aceddfe14fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:01.094802 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afeef12-6ac0-43f5-8a27-aceddfe14fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:01.095080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afeef12-6ac0-43f5-8a27-aceddfe14fe9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:01.095823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afeef12-6ac0-43f5-8a27-aceddfe14fe9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:07:01.096129 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afeef12-6ac0-43f5-8a27-aceddfe14fe9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:01.097774 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afeef12-6ac0-43f5-8a27-aceddfe14fe9 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:01.106559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1100/2176] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:07:01.111380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1077/2177] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 22:07:01.116936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5468e7-a066-434b-a202-a1b20ff6b75b None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:01.117077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afeef12-6ac0-43f5-8a27-aceddfe14fe9 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:01.119375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8afeef12-6ac0-43f5-8a27-aceddfe14fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:01.127092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5468e7-a066-434b-a202-a1b20ff6b75b None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:01.132415 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:01.135249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf02d2-6da5-4cab-a867-01222201d7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:01.135872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf02d2-6da5-4cab-a867-01222201d7c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:01.136164 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf02d2-6da5-4cab-a867-01222201d7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:01.136451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf02d2-6da5-4cab-a867-01222201d7c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:01.137215 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf02d2-6da5-4cab-a867-01222201d7c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:01.141120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf02d2-6da5-4cab-a867-01222201d7c1 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:01.189756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1101/2178] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:07:01 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:07:01.196525 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf02d2-6da5-4cab-a867-01222201d7c1 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:01.197047 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afeef12-6ac0-43f5-8a27-aceddfe14fe9 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:01.250184 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb793e0-077f-49f0-a746-531c3b07da74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:01.250560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:01.250849 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb793e0-077f-49f0-a746-531c3b07da74 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:01.251136 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb793e0-077f-49f0-a746-531c3b07da74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:01.251420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb793e0-077f-49f0-a746-531c3b07da74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:01.269625 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf02d2-6da5-4cab-a867-01222201d7c1 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:01.274865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd35b3e-a0cb-4c1a-a803-64bad0d3102f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:01.275586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd35b3e-a0cb-4c1a-a803-64bad0d3102f admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:01.275871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd35b3e-a0cb-4c1a-a803-64bad0d3102f admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:01.276152 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd35b3e-a0cb-4c1a-a803-64bad0d3102f admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:01.276920 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd35b3e-a0cb-4c1a-a803-64bad0d3102f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:01.278923 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd35b3e-a0cb-4c1a-a803-64bad0d3102f admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:01.299822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1102/2179] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:07:01 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:07:01.304992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd35b3e-a0cb-4c1a-a803-64bad0d3102f admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:01.318024 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd35b3e-a0cb-4c1a-a803-64bad0d3102f admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:01.318235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb793e0-077f-49f0-a746-531c3b07da74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:01.320443 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb793e0-077f-49f0-a746-531c3b07da74 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:01.324255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8c5eb-09a6-483c-875c-abae4909d6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:01.324987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8c5eb-09a6-483c-875c-abae4909d6da admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:01.325273 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8c5eb-09a6-483c-875c-abae4909d6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:01.325549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8c5eb-09a6-483c-875c-abae4909d6da admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:01.326289 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8c5eb-09a6-483c-875c-abae4909d6da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:01.327905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8c5eb-09a6-483c-875c-abae4909d6da admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:01.331310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1078/2180] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:07:01.339739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1103/2181] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:07:01.344018 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb793e0-077f-49f0-a746-531c3b07da74 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:01.354023 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb793e0-077f-49f0-a746-531c3b07da74 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:01.359279 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01e29e-18ba-4e66-be9b-a3b336d6bd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:01.359947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01e29e-18ba-4e66-be9b-a3b336d6bd16 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:01.360233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01e29e-18ba-4e66-be9b-a3b336d6bd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:01.360635 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01e29e-18ba-4e66-be9b-a3b336d6bd16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:01.361416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01e29e-18ba-4e66-be9b-a3b336d6bd16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:01.363033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01e29e-18ba-4e66-be9b-a3b336d6bd16 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:01.377899 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1079/2182] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:07:01.379462 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8c5eb-09a6-483c-875c-abae4909d6da admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:01.390946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8c5eb-09a6-483c-875c-abae4909d6da admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:01.396396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd772a-8f76-4f10-9ece-2f5cc315fff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:01.397902 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd772a-8f76-4f10-9ece-2f5cc315fff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:01.398188 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd772a-8f76-4f10-9ece-2f5cc315fff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:01.398945 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd772a-8f76-4f10-9ece-2f5cc315fff2 admin admin] PATH_INFO: `/v3/projects/cc7dd17d21604b1e91c57cf999680743/users/1bef8cc467d34666bde95d1d830b1061/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:01.419183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd772a-8f76-4f10-9ece-2f5cc315fff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc7dd17d21604b1e91c57cf999680743, user_id=1bef8cc467d34666bde95d1d830b1061, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:01.422457 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd772a-8f76-4f10-9ece-2f5cc315fff2 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:01.438046 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1104/2183] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:07:01 2022] PUT /identity/v3/projects/cc7dd17d21604b1e91c57cf999680743/users/1bef8cc467d34666bde95d1d830b1061/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:01.451612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33bd21-9e58-4ed6-a9d6-511d08784b92 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:01.451925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33bd21-9e58-4ed6-a9d6-511d08784b92 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:01.452205 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33bd21-9e58-4ed6-a9d6-511d08784b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:01.481801 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-8e33bd21-9e58-4ed6-a9d6-511d08784b92 None None] MFA Rules not processed for user `1bef8cc467d34666bde95d1d830b1061`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:01.483051 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8e33bd21-9e58-4ed6-a9d6-511d08784b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:01.573723 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:01.586022 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1080/2184] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:01.599053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd772a-8f76-4f10-9ece-2f5cc315fff2 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:01.605734 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d3bd772a-8f76-4f10-9ece-2f5cc315fff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:01.671992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd772a-8f76-4f10-9ece-2f5cc315fff2 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:01.676177 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79a6c9-ec77-4f56-9576-4809f3737052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:01.676832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79a6c9-ec77-4f56-9576-4809f3737052 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:01.677153 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79a6c9-ec77-4f56-9576-4809f3737052 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:01.677619 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79a6c9-ec77-4f56-9576-4809f3737052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:01.688161 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79a6c9-ec77-4f56-9576-4809f3737052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:01.689682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79a6c9-ec77-4f56-9576-4809f3737052 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:01.701405 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1105/2185] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:07:02.557313 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33bd21-9e58-4ed6-a9d6-511d08784b92 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:02.559143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8e33bd21-9e58-4ed6-a9d6-511d08784b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:02.643247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33bd21-9e58-4ed6-a9d6-511d08784b92 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:02.648282 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6d99e-ec6c-4dea-a4e0-ed5f1613c02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:02.649225 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6d99e-ec6c-4dea-a4e0-ed5f1613c02e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:02.649511 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6d99e-ec6c-4dea-a4e0-ed5f1613c02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:02.649803 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6d99e-ec6c-4dea-a4e0-ed5f1613c02e admin admin] PATH_INFO: `/v3/users/355eec92657542739ecfc08d307379e5` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:02.651297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd6d99e-ec6c-4dea-a4e0-ed5f1613c02e admin admin] RBAC: Authorizing `identity:delete_user(user_id=355eec92657542739ecfc08d307379e5)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:02.653415 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd6d99e-ec6c-4dea-a4e0-ed5f1613c02e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:02.712904 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-bdd6d99e-ec6c-4dea-a4e0-ed5f1613c02e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '355eec92657542739ecfc08d307379e5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:02.722513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-bdd6d99e-ec6c-4dea-a4e0-ed5f1613c02e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '355eec92657542739ecfc08d307379e5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:02.732369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-bdd6d99e-ec6c-4dea-a4e0-ed5f1613c02e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '355eec92657542739ecfc08d307379e5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:02.733076 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-bdd6d99e-ec6c-4dea-a4e0-ed5f1613c02e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '355eec92657542739ecfc08d307379e5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:02.733721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-bdd6d99e-ec6c-4dea-a4e0-ed5f1613c02e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '355eec92657542739ecfc08d307379e5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:02.760013 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1081/2186] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:07:02 2022] DELETE /identity/v3/users/355eec92657542739ecfc08d307379e5 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:02.761834 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930fc53-bdab-4768-92a6-00fb73938364 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:02.762323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930fc53-bdab-4768-92a6-00fb73938364 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:02.762405 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930fc53-bdab-4768-92a6-00fb73938364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:02.783419 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-c930fc53-bdab-4768-92a6-00fb73938364 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:02.785075 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c930fc53-bdab-4768-92a6-00fb73938364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:02.875647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1106/2187] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:02.883303 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6d99e-ec6c-4dea-a4e0-ed5f1613c02e admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:02.884949 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-bdd6d99e-ec6c-4dea-a4e0-ed5f1613c02e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:02.958187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6d99e-ec6c-4dea-a4e0-ed5f1613c02e admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:02.973194 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e009183-c635-49c4-b089-f13a66c41351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:02.973421 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e009183-c635-49c4-b089-f13a66c41351 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:02.973712 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e009183-c635-49c4-b089-f13a66c41351 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:02.975312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e009183-c635-49c4-b089-f13a66c41351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:02.985434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e009183-c635-49c4-b089-f13a66c41351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:02.986881 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e009183-c635-49c4-b089-f13a66c41351 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:03.003849 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1082/2188] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:07:03.404561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930fc53-bdab-4768-92a6-00fb73938364 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:03.415217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c930fc53-bdab-4768-92a6-00fb73938364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:03.439041 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930fc53-bdab-4768-92a6-00fb73938364 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:03.448638 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:03.449776 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:03.450116 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:03.450404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c admin admin] PATH_INFO: `/v3/projects/9eea115b174e45a697351ae2e77b9dbb` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:03.473408 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eea115b174e45a697351ae2e77b9dbb)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:03.476075 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:03.540174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:03.562166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eea115b174e45a697351ae2e77b9dbb', 'request_id': 'req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:03.583955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eea115b174e45a697351ae2e77b9dbb', 'request_id': 'req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:03.596764 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eea115b174e45a697351ae2e77b9dbb', 'request_id': 'req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:03.614988 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c admin admin] The token cache is being invalidate because project 9eea115b174e45a697351ae2e77b9dbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:07:03.616150 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:03.616947 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:03.618951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1107/2189] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:07:03 2022] DELETE /identity/v3/projects/9eea115b174e45a697351ae2e77b9dbb => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:03.622873 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e009183-c635-49c4-b089-f13a66c41351 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:03.640205 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e009183-c635-49c4-b089-f13a66c41351 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:03.657131 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:03.727270 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a812f0a-7c6d-4bae-9e95-f658a7aaf31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:03.728334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a812f0a-7c6d-4bae-9e95-f658a7aaf31b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:03.728621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a812f0a-7c6d-4bae-9e95-f658a7aaf31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:03.728928 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a812f0a-7c6d-4bae-9e95-f658a7aaf31b admin admin] PATH_INFO: `/v3/users/7859d4d1f41a4ac1b826295c786ebebb` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:03.730768 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a812f0a-7c6d-4bae-9e95-f658a7aaf31b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7859d4d1f41a4ac1b826295c786ebebb)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:03.732753 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a812f0a-7c6d-4bae-9e95-f658a7aaf31b admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:03.784975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1a812f0a-7c6d-4bae-9e95-f658a7aaf31b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7859d4d1f41a4ac1b826295c786ebebb'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:03.792755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1a812f0a-7c6d-4bae-9e95-f658a7aaf31b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7859d4d1f41a4ac1b826295c786ebebb'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:03.800944 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1a812f0a-7c6d-4bae-9e95-f658a7aaf31b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7859d4d1f41a4ac1b826295c786ebebb'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:03.801615 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1a812f0a-7c6d-4bae-9e95-f658a7aaf31b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7859d4d1f41a4ac1b826295c786ebebb'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:03.802243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1a812f0a-7c6d-4bae-9e95-f658a7aaf31b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7859d4d1f41a4ac1b826295c786ebebb'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:03.823806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1083/2190] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:07:03 2022] DELETE /identity/v3/users/7859d4d1f41a4ac1b826295c786ebebb => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:04.321314 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:04.325308 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:04.349056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e6bcb-78f6-433e-9168-6b4b26ab1d0c admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:04.420270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de649fa-e0b6-4b05-bf4a-dcfcb331e465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:04.421473 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de649fa-e0b6-4b05-bf4a-dcfcb331e465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:04.421763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de649fa-e0b6-4b05-bf4a-dcfcb331e465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:04.422051 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de649fa-e0b6-4b05-bf4a-dcfcb331e465 admin admin] PATH_INFO: `/v3/projects/9e813a59e84b4339a512d02b1096afa8` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:04.423377 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de649fa-e0b6-4b05-bf4a-dcfcb331e465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e813a59e84b4339a512d02b1096afa8)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:04.425269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de649fa-e0b6-4b05-bf4a-dcfcb331e465 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:04.480377 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-5de649fa-e0b6-4b05-bf4a-dcfcb331e465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e813a59e84b4339a512d02b1096afa8', 'request_id': 'req-5de649fa-e0b6-4b05-bf4a-dcfcb331e465'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:04.486945 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-5de649fa-e0b6-4b05-bf4a-dcfcb331e465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e813a59e84b4339a512d02b1096afa8', 'request_id': 'req-5de649fa-e0b6-4b05-bf4a-dcfcb331e465'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:04.497345 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-5de649fa-e0b6-4b05-bf4a-dcfcb331e465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e813a59e84b4339a512d02b1096afa8', 'request_id': 'req-5de649fa-e0b6-4b05-bf4a-dcfcb331e465'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:04.512519 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-5de649fa-e0b6-4b05-bf4a-dcfcb331e465 admin admin] The token cache is being invalidate because project 9e813a59e84b4339a512d02b1096afa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:07:04.512871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-5de649fa-e0b6-4b05-bf4a-dcfcb331e465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:04.513645 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-5de649fa-e0b6-4b05-bf4a-dcfcb331e465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:04.515701 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1108/2191] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:07:04 2022] DELETE /identity/v3/projects/9e813a59e84b4339a512d02b1096afa8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:04.520968 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9f2ed-34fa-42aa-9e74-e80b0e7fb1b9 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:04.521264 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9f2ed-34fa-42aa-9e74-e80b0e7fb1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:04.521892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9f2ed-34fa-42aa-9e74-e80b0e7fb1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:04.541667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-8ec9f2ed-34fa-42aa-9e74-e80b0e7fb1b9 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:04.543011 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8ec9f2ed-34fa-42aa-9e74-e80b0e7fb1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:04.618177 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1084/2192] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:07:04.626314 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160db92-96ff-4a60-a4c6-0c5b00bd8521 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:04.626899 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160db92-96ff-4a60-a4c6-0c5b00bd8521 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:04.626899 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160db92-96ff-4a60-a4c6-0c5b00bd8521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:04.654011 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-d160db92-96ff-4a60-a4c6-0c5b00bd8521 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:04.655698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d160db92-96ff-4a60-a4c6-0c5b00bd8521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:04.669930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1109/2193] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:04.671289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9f2ed-34fa-42aa-9e74-e80b0e7fb1b9 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:04.687676 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9f2ed-34fa-42aa-9e74-e80b0e7fb1b9 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:04.693636 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d85be-7d3a-4a20-97ad-7802a313c8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:04.694892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d85be-7d3a-4a20-97ad-7802a313c8fb admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:04.695379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d85be-7d3a-4a20-97ad-7802a313c8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:04.695965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d85be-7d3a-4a20-97ad-7802a313c8fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:04.696891 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d85be-7d3a-4a20-97ad-7802a313c8fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:07:04.697352 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d85be-7d3a-4a20-97ad-7802a313c8fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:04.699588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d85be-7d3a-4a20-97ad-7802a313c8fb admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:04.714181 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1085/2194] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:07:04.718784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160db92-96ff-4a60-a4c6-0c5b00bd8521 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:04.729783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160db92-96ff-4a60-a4c6-0c5b00bd8521 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:04.736496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cad14-e5e6-4ea2-8ce8-6923d2bee018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:04.737008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cad14-e5e6-4ea2-8ce8-6923d2bee018 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:04.737681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cad14-e5e6-4ea2-8ce8-6923d2bee018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:04.737681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cad14-e5e6-4ea2-8ce8-6923d2bee018 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:04.738644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1cad14-e5e6-4ea2-8ce8-6923d2bee018 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:04.740776 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1cad14-e5e6-4ea2-8ce8-6923d2bee018 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:04.777946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1110/2195] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:07:04 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:07:04.782483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d85be-7d3a-4a20-97ad-7802a313c8fb admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:04.799353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d85be-7d3a-4a20-97ad-7802a313c8fb admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:04.836533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:04.903645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f6a90-9a15-4fbd-bae4-9461ea7d7133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:04.905794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f6a90-9a15-4fbd-bae4-9461ea7d7133 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:04.906631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f6a90-9a15-4fbd-bae4-9461ea7d7133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:04.906631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f6a90-9a15-4fbd-bae4-9461ea7d7133 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:04.907216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f6a90-9a15-4fbd-bae4-9461ea7d7133 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:04.909292 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f6a90-9a15-4fbd-bae4-9461ea7d7133 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:04.940377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1086/2196] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:07:04 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:07:04.941746 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cad14-e5e6-4ea2-8ce8-6923d2bee018 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:04.955657 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cad14-e5e6-4ea2-8ce8-6923d2bee018 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:04.962980 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df9e20-1cfd-4fa7-91f5-e0a046cc1e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:04.963617 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df9e20-1cfd-4fa7-91f5-e0a046cc1e23 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:04.964470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df9e20-1cfd-4fa7-91f5-e0a046cc1e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:04.964470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df9e20-1cfd-4fa7-91f5-e0a046cc1e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:04.964934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df9e20-1cfd-4fa7-91f5-e0a046cc1e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:04.966956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df9e20-1cfd-4fa7-91f5-e0a046cc1e23 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:04.984763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1111/2197] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:07:04.986222 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f6a90-9a15-4fbd-bae4-9461ea7d7133 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:05.000935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f6a90-9a15-4fbd-bae4-9461ea7d7133 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:05.005977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e8458-64e1-4b7d-9959-52be964f2aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:05.007819 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e8458-64e1-4b7d-9959-52be964f2aca admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:05.008113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e8458-64e1-4b7d-9959-52be964f2aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:05.008387 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e8458-64e1-4b7d-9959-52be964f2aca admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:05.009357 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e8458-64e1-4b7d-9959-52be964f2aca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:05.011007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e8458-64e1-4b7d-9959-52be964f2aca admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:05.025397 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1087/2198] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:07:05.029582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df9e20-1cfd-4fa7-91f5-e0a046cc1e23 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:05.040660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df9e20-1cfd-4fa7-91f5-e0a046cc1e23 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:05.045681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe46b0-ed18-4289-9fa2-3dcb4956038d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:05.046574 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe46b0-ed18-4289-9fa2-3dcb4956038d admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:05.046854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe46b0-ed18-4289-9fa2-3dcb4956038d admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:05.047130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe46b0-ed18-4289-9fa2-3dcb4956038d admin admin] PATH_INFO: `/v3/projects/7b528fe3a3be4b6eb49de69655952e36/users/0ee70760d01048adb24e92a12094d1fb/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:05.068146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:05.068146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe46b0-ed18-4289-9fa2-3dcb4956038d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b528fe3a3be4b6eb49de69655952e36, user_id=0ee70760d01048adb24e92a12094d1fb, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:05.070071 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe46b0-ed18-4289-9fa2-3dcb4956038d admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:05.088807 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1112/2199] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:07:05 2022] PUT /identity/v3/projects/7b528fe3a3be4b6eb49de69655952e36/users/0ee70760d01048adb24e92a12094d1fb/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:05.094318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fff18-8ab5-4c4c-88a6-423dddb2459a None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:05.095277 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fff18-8ab5-4c4c-88a6-423dddb2459a None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:05.095561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fff18-8ab5-4c4c-88a6-423dddb2459a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:05.116183 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-610fff18-8ab5-4c4c-88a6-423dddb2459a None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:05.117448 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-610fff18-8ab5-4c4c-88a6-423dddb2459a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:05.189975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1088/2200] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:05.200146 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe46b0-ed18-4289-9fa2-3dcb4956038d admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:05.202630 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-f3fe46b0-ed18-4289-9fa2-3dcb4956038d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:05.297915 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe46b0-ed18-4289-9fa2-3dcb4956038d admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:05.302004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0708c-5449-4b64-9339-971c53fd895f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:05.302664 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0708c-5449-4b64-9339-971c53fd895f service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:05.303798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0708c-5449-4b64-9339-971c53fd895f service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:05.304092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0708c-5449-4b64-9339-971c53fd895f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:05.315298 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0708c-5449-4b64-9339-971c53fd895f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:05.317952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0708c-5449-4b64-9339-971c53fd895f service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:05.332502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1113/2201] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:07:06.183848 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47474645-986e-449b-8129-76f3cad1ef8b None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:06.184390 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47474645-986e-449b-8129-76f3cad1ef8b None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:06.184390 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47474645-986e-449b-8129-76f3cad1ef8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:06.229128 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-47474645-986e-449b-8129-76f3cad1ef8b None None] MFA Rules not processed for user `0ee70760d01048adb24e92a12094d1fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:06.232079 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-47474645-986e-449b-8129-76f3cad1ef8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:06.329781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0708c-5449-4b64-9339-971c53fd895f service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:06.330036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1089/2202] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:06.341925 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0708c-5449-4b64-9339-971c53fd895f service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:06.351843 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eee03d-9bde-4a48-80d1-ffde529b7e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:06.353195 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eee03d-9bde-4a48-80d1-ffde529b7e83 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:06.353671 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eee03d-9bde-4a48-80d1-ffde529b7e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:06.354109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eee03d-9bde-4a48-80d1-ffde529b7e83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:06.354972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eee03d-9bde-4a48-80d1-ffde529b7e83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:06.357786 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eee03d-9bde-4a48-80d1-ffde529b7e83 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:06.393027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1114/2203] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:07:06 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:07:06.397305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47474645-986e-449b-8129-76f3cad1ef8b None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:06.410686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47474645-986e-449b-8129-76f3cad1ef8b None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:06.451935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:06.495998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67371f66-1203-4fb0-82f4-5801d12c2050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:06.497185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67371f66-1203-4fb0-82f4-5801d12c2050 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:06.497532 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67371f66-1203-4fb0-82f4-5801d12c2050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:06.498349 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67371f66-1203-4fb0-82f4-5801d12c2050 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:06.498530 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67371f66-1203-4fb0-82f4-5801d12c2050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:06.501247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67371f66-1203-4fb0-82f4-5801d12c2050 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:06.530282 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1090/2204] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:07:06 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:07:06.533955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eee03d-9bde-4a48-80d1-ffde529b7e83 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:06.548361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eee03d-9bde-4a48-80d1-ffde529b7e83 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:06.555835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273052b-0230-4a5c-a05b-792318302d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:06.556829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273052b-0230-4a5c-a05b-792318302d18 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:06.557255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273052b-0230-4a5c-a05b-792318302d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:06.557681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273052b-0230-4a5c-a05b-792318302d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:06.558611 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273052b-0230-4a5c-a05b-792318302d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:06.561251 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273052b-0230-4a5c-a05b-792318302d18 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:06.582560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1115/2205] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:07:06.582977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67371f66-1203-4fb0-82f4-5801d12c2050 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:06.597908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67371f66-1203-4fb0-82f4-5801d12c2050 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:06.605194 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53811ab-184d-4b42-bbaf-5f38c7e12f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:06.606125 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53811ab-184d-4b42-bbaf-5f38c7e12f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:06.606630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53811ab-184d-4b42-bbaf-5f38c7e12f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:06.606912 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53811ab-184d-4b42-bbaf-5f38c7e12f9a admin admin] PATH_INFO: `/v3/projects/264ee15846074e94b675502ac4ac025a/users/be0b66df14bd41c292e4058877d1019d/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:06.627279 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53811ab-184d-4b42-bbaf-5f38c7e12f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=264ee15846074e94b675502ac4ac025a, user_id=be0b66df14bd41c292e4058877d1019d, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:06.630572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53811ab-184d-4b42-bbaf-5f38c7e12f9a admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:06.650769 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1091/2206] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:07:06 2022] PUT /identity/v3/projects/264ee15846074e94b675502ac4ac025a/users/be0b66df14bd41c292e4058877d1019d/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:07:06.654401 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273052b-0230-4a5c-a05b-792318302d18 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:06.668107 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273052b-0230-4a5c-a05b-792318302d18 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:06.755061 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f618f-0b77-42cf-b1a3-44f3bad1a797 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:06.755351 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f618f-0b77-42cf-b1a3-44f3bad1a797 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:06.755632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f618f-0b77-42cf-b1a3-44f3bad1a797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:06.762249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43612da0-94a8-4c84-a0ae-5b2838facf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:06.764441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43612da0-94a8-4c84-a0ae-5b2838facf0e admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:06.765242 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43612da0-94a8-4c84-a0ae-5b2838facf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:06.765242 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43612da0-94a8-4c84-a0ae-5b2838facf0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:06.766207 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43612da0-94a8-4c84-a0ae-5b2838facf0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:06.767875 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43612da0-94a8-4c84-a0ae-5b2838facf0e admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:06.785102 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-f16f618f-0b77-42cf-b1a3-44f3bad1a797 None None] MFA Rules not processed for user `309de655c8d64223a9c2d75ff5618411`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:06.786380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-f16f618f-0b77-42cf-b1a3-44f3bad1a797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:06.791102 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1116/2207] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:07:06.792552 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43612da0-94a8-4c84-a0ae-5b2838facf0e admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:06.806992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43612da0-94a8-4c84-a0ae-5b2838facf0e admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:06.812019 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd876d-a40f-45f5-ac2c-d89ed0b8909d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:06.813111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd876d-a40f-45f5-ac2c-d89ed0b8909d admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:06.813628 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd876d-a40f-45f5-ac2c-d89ed0b8909d admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:06.814043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd876d-a40f-45f5-ac2c-d89ed0b8909d admin admin] PATH_INFO: `/v3/projects/264ee15846074e94b675502ac4ac025a/users/be0b66df14bd41c292e4058877d1019d/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:06.816314 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd876d-a40f-45f5-ac2c-d89ed0b8909d admin admin] RBAC: Authorizing `identity:create_grant(project_id=264ee15846074e94b675502ac4ac025a, user_id=be0b66df14bd41c292e4058877d1019d, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:06.820271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd876d-a40f-45f5-ac2c-d89ed0b8909d admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:06.832264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1117/2208] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:07:06 2022] PUT /identity/v3/projects/264ee15846074e94b675502ac4ac025a/users/be0b66df14bd41c292e4058877d1019d/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:06.878404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1092/2209] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:07:06.891932 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c18c4-6e6a-46cf-9c3d-e2165ff90042 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:06.892225 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c18c4-6e6a-46cf-9c3d-e2165ff90042 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:06.892505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c18c4-6e6a-46cf-9c3d-e2165ff90042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:06.914669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-394c18c4-6e6a-46cf-9c3d-e2165ff90042 None None] MFA Rules not processed for user `309de655c8d64223a9c2d75ff5618411`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:06.915958 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-394c18c4-6e6a-46cf-9c3d-e2165ff90042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:06.989966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:07.017527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1118/2210] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:07.029106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f618f-0b77-42cf-b1a3-44f3bad1a797 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:07.031003 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-f16f618f-0b77-42cf-b1a3-44f3bad1a797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:07.141643 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f618f-0b77-42cf-b1a3-44f3bad1a797 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:07.167218 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:07.243998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e80c8-9e7d-4431-b7a6-814ea86fd1c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:07.244873 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e80c8-9e7d-4431-b7a6-814ea86fd1c4 service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:07.245231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e80c8-9e7d-4431-b7a6-814ea86fd1c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:07.245611 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e80c8-9e7d-4431-b7a6-814ea86fd1c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:07.253975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e80c8-9e7d-4431-b7a6-814ea86fd1c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:07.256055 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e80c8-9e7d-4431-b7a6-814ea86fd1c4 service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:07.272954 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1093/2211] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:07:08.055401 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ebb12-fe08-404b-be8d-85945c76eced None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:08.055998 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ebb12-fe08-404b-be8d-85945c76eced None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:08.056204 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ebb12-fe08-404b-be8d-85945c76eced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:08.092043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-3b9ebb12-fe08-404b-be8d-85945c76eced None None] MFA Rules not processed for user `be0b66df14bd41c292e4058877d1019d`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:08.093637 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-3b9ebb12-fe08-404b-be8d-85945c76eced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:08.197952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1119/2212] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:08.217774 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e80c8-9e7d-4431-b7a6-814ea86fd1c4 service nova] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:08.219921 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-3f3e80c8-9e7d-4431-b7a6-814ea86fd1c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:08.304368 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c973d5e-1d37-4905-a332-54620b572e90 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:08.304655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c973d5e-1d37-4905-a332-54620b572e90 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:08.304963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c973d5e-1d37-4905-a332-54620b572e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:08.327436 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e80c8-9e7d-4431-b7a6-814ea86fd1c4 service nova] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:08.332691 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489949d2-bd68-46ac-a5f0-f5efba944e5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:08.334643 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489949d2-bd68-46ac-a5f0-f5efba944e5c service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:08.337259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489949d2-bd68-46ac-a5f0-f5efba944e5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:08.337761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489949d2-bd68-46ac-a5f0-f5efba944e5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:08.351778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-3c973d5e-1d37-4905-a332-54620b572e90 None None] MFA Rules not processed for user `309de655c8d64223a9c2d75ff5618411`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:08.357367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-3c973d5e-1d37-4905-a332-54620b572e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:08.381283 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1120/2213] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:07:08.447096 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489949d2-bd68-46ac-a5f0-f5efba944e5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:08.449673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489949d2-bd68-46ac-a5f0-f5efba944e5c service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:08.464430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1094/2214] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:07:12.120983 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c973d5e-1d37-4905-a332-54620b572e90 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:12.131678 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c973d5e-1d37-4905-a332-54620b572e90 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:12.135420 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0c2b3-d821-44a7-943c-4ad08f2bdc79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:12.136066 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0c2b3-d821-44a7-943c-4ad08f2bdc79 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:12.136363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0c2b3-d821-44a7-943c-4ad08f2bdc79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:12.136640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0c2b3-d821-44a7-943c-4ad08f2bdc79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:12.149157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0c2b3-d821-44a7-943c-4ad08f2bdc79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:12.150681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0c2b3-d821-44a7-943c-4ad08f2bdc79 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:12.160548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1121/2215] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:12 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:07:14.204835 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489949d2-bd68-46ac-a5f0-f5efba944e5c service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:14.206922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-489949d2-bd68-46ac-a5f0-f5efba944e5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:14.307295 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489949d2-bd68-46ac-a5f0-f5efba944e5c service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:14.327875 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b4470-8722-4f15-8341-477083d2ded5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:14.329118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b4470-8722-4f15-8341-477083d2ded5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:14.329472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b4470-8722-4f15-8341-477083d2ded5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:14.329850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b4470-8722-4f15-8341-477083d2ded5 admin admin] PATH_INFO: `/v3/users/72a9a7fdb2e8491a94257215cda417b1` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:14.332356 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b4470-8722-4f15-8341-477083d2ded5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72a9a7fdb2e8491a94257215cda417b1)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:14.335478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b4470-8722-4f15-8341-477083d2ded5 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:14.342362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:14.425284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9a6b4470-8722-4f15-8341-477083d2ded5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72a9a7fdb2e8491a94257215cda417b1'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:14.435850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9a6b4470-8722-4f15-8341-477083d2ded5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72a9a7fdb2e8491a94257215cda417b1'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:14.447405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9a6b4470-8722-4f15-8341-477083d2ded5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72a9a7fdb2e8491a94257215cda417b1'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:14.448444 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9a6b4470-8722-4f15-8341-477083d2ded5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72a9a7fdb2e8491a94257215cda417b1'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:14.449376 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9a6b4470-8722-4f15-8341-477083d2ded5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72a9a7fdb2e8491a94257215cda417b1'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:14.477477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1095/2216] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:07:14 2022] DELETE /identity/v3/users/72a9a7fdb2e8491a94257215cda417b1 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:14.482771 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad6817-7f8f-4d78-85bd-c5a2e5c97f74 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:14.483754 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad6817-7f8f-4d78-85bd-c5a2e5c97f74 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:14.484301 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad6817-7f8f-4d78-85bd-c5a2e5c97f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:14.507446 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-4fad6817-7f8f-4d78-85bd-c5a2e5c97f74 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:14.508839 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-4fad6817-7f8f-4d78-85bd-c5a2e5c97f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:14.574880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1122/2217] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:07:14.585914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b4470-8722-4f15-8341-477083d2ded5 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:14.587393 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-9a6b4470-8722-4f15-8341-477083d2ded5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:14.678379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b4470-8722-4f15-8341-477083d2ded5 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:14.683076 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae444b4-32c0-4362-beeb-12bd65074bea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:14.683742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae444b4-32c0-4362-beeb-12bd65074bea service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:14.684024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae444b4-32c0-4362-beeb-12bd65074bea service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:14.684423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae444b4-32c0-4362-beeb-12bd65074bea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:14.696987 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae444b4-32c0-4362-beeb-12bd65074bea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:14.698170 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae444b4-32c0-4362-beeb-12bd65074bea service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:14.709217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1096/2218] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:07:16.044091 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad6817-7f8f-4d78-85bd-c5a2e5c97f74 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:16.045783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-4fad6817-7f8f-4d78-85bd-c5a2e5c97f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:16.062348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad6817-7f8f-4d78-85bd-c5a2e5c97f74 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:16.068658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e443d1-b574-4ea5-b35d-686e066c7b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:16.070247 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e443d1-b574-4ea5-b35d-686e066c7b98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:16.070597 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e443d1-b574-4ea5-b35d-686e066c7b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:16.070858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e443d1-b574-4ea5-b35d-686e066c7b98 admin admin] PATH_INFO: `/v3/projects/c53e7071c8e147ce9a5e239e773b8dcc` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:16.072156 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e443d1-b574-4ea5-b35d-686e066c7b98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c53e7071c8e147ce9a5e239e773b8dcc)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:16.074716 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e443d1-b574-4ea5-b35d-686e066c7b98 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:16.129974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:16.132543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-10e443d1-b574-4ea5-b35d-686e066c7b98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c53e7071c8e147ce9a5e239e773b8dcc', 'request_id': 'req-10e443d1-b574-4ea5-b35d-686e066c7b98'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:16.139785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-10e443d1-b574-4ea5-b35d-686e066c7b98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c53e7071c8e147ce9a5e239e773b8dcc', 'request_id': 'req-10e443d1-b574-4ea5-b35d-686e066c7b98'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:16.147451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-10e443d1-b574-4ea5-b35d-686e066c7b98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c53e7071c8e147ce9a5e239e773b8dcc', 'request_id': 'req-10e443d1-b574-4ea5-b35d-686e066c7b98'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:16.159626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-10e443d1-b574-4ea5-b35d-686e066c7b98 admin admin] The token cache is being invalidate because project c53e7071c8e147ce9a5e239e773b8dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:07:16.159914 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-10e443d1-b574-4ea5-b35d-686e066c7b98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:16.160583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-10e443d1-b574-4ea5-b35d-686e066c7b98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:16.162663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1123/2219] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:07:16 2022] DELETE /identity/v3/projects/c53e7071c8e147ce9a5e239e773b8dcc => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:16.167584 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4117ee7-1a82-4080-afad-b5fa62de9206 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:16.168090 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4117ee7-1a82-4080-afad-b5fa62de9206 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:16.168181 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4117ee7-1a82-4080-afad-b5fa62de9206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:16.188837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-d4117ee7-1a82-4080-afad-b5fa62de9206 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:16.190045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d4117ee7-1a82-4080-afad-b5fa62de9206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:16.268204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1097/2220] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:07:16.276594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f7fca-4e08-4d44-9248-97d4c7d9ffe1 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:16.276955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f7fca-4e08-4d44-9248-97d4c7d9ffe1 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:16.277324 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f7fca-4e08-4d44-9248-97d4c7d9ffe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:16.297569 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-8d7f7fca-4e08-4d44-9248-97d4c7d9ffe1 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:16.298812 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8d7f7fca-4e08-4d44-9248-97d4c7d9ffe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:16.307807 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1124/2221] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:16.312320 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4117ee7-1a82-4080-afad-b5fa62de9206 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:16.338762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4117ee7-1a82-4080-afad-b5fa62de9206 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:16.350811 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e21ad6-0258-41cb-b781-99b4b4d0780b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:16.351508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e21ad6-0258-41cb-b781-99b4b4d0780b admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:16.351796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e21ad6-0258-41cb-b781-99b4b4d0780b admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:16.352076 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e21ad6-0258-41cb-b781-99b4b4d0780b admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:16.352856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e21ad6-0258-41cb-b781-99b4b4d0780b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:07:16.353171 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e21ad6-0258-41cb-b781-99b4b4d0780b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:16.355091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e21ad6-0258-41cb-b781-99b4b4d0780b admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:16.407027 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1098/2222] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:07:16.413374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f7fca-4e08-4d44-9248-97d4c7d9ffe1 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:16.441697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f7fca-4e08-4d44-9248-97d4c7d9ffe1 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:16.448975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e2c74-5deb-4726-842f-986aef303505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:16.449655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e2c74-5deb-4726-842f-986aef303505 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:16.450117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e2c74-5deb-4726-842f-986aef303505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:16.474879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e2c74-5deb-4726-842f-986aef303505 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:16.480316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e2c74-5deb-4726-842f-986aef303505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:16.482935 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e2c74-5deb-4726-842f-986aef303505 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:16.536443 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1125/2223] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:07:16 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:07:16.541522 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e21ad6-0258-41cb-b781-99b4b4d0780b admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:16.554646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e21ad6-0258-41cb-b781-99b4b4d0780b admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:16.566012 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:16.645158 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede5a01-db51-4379-a9aa-9ecdf9c0e0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:16.646080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede5a01-db51-4379-a9aa-9ecdf9c0e0e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:16.646436 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede5a01-db51-4379-a9aa-9ecdf9c0e0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:16.646842 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede5a01-db51-4379-a9aa-9ecdf9c0e0e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:16.647912 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede5a01-db51-4379-a9aa-9ecdf9c0e0e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:16.649900 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede5a01-db51-4379-a9aa-9ecdf9c0e0e0 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:16.682989 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1099/2224] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:07:16 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:07:16.697007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e2c74-5deb-4726-842f-986aef303505 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:16.713151 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e2c74-5deb-4726-842f-986aef303505 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:16.719213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99989e2-20df-42a8-bf93-be50aa1321ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:16.720349 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99989e2-20df-42a8-bf93-be50aa1321ce admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:16.720744 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99989e2-20df-42a8-bf93-be50aa1321ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:16.721213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99989e2-20df-42a8-bf93-be50aa1321ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:16.722331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99989e2-20df-42a8-bf93-be50aa1321ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:16.725277 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99989e2-20df-42a8-bf93-be50aa1321ce admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:16.744276 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1126/2225] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:07:16.745530 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede5a01-db51-4379-a9aa-9ecdf9c0e0e0 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:16.756124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede5a01-db51-4379-a9aa-9ecdf9c0e0e0 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:16.761034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea06001-a377-461a-a4fd-7878c5968b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:16.761620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea06001-a377-461a-a4fd-7878c5968b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:16.761942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea06001-a377-461a-a4fd-7878c5968b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:16.762166 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea06001-a377-461a-a4fd-7878c5968b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:16.762896 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea06001-a377-461a-a4fd-7878c5968b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:16.764498 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea06001-a377-461a-a4fd-7878c5968b31 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:16.775348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1100/2226] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:07:16.779391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99989e2-20df-42a8-bf93-be50aa1321ce admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:16.790999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99989e2-20df-42a8-bf93-be50aa1321ce admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:16.796047 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f574a02-2a35-48cd-99e2-20fd0d0e7839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:16.796984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f574a02-2a35-48cd-99e2-20fd0d0e7839 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:16.797275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f574a02-2a35-48cd-99e2-20fd0d0e7839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:16.797563 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f574a02-2a35-48cd-99e2-20fd0d0e7839 admin admin] PATH_INFO: `/v3/projects/39f88660bffb4d17ba8b5a7a00c3f6d3/users/4c137f9412de4914b77732e1ac29ec36/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:16.813525 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f574a02-2a35-48cd-99e2-20fd0d0e7839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39f88660bffb4d17ba8b5a7a00c3f6d3, user_id=4c137f9412de4914b77732e1ac29ec36, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:16.816187 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f574a02-2a35-48cd-99e2-20fd0d0e7839 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:16.830430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1127/2227] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:07:16 2022] PUT /identity/v3/projects/39f88660bffb4d17ba8b5a7a00c3f6d3/users/4c137f9412de4914b77732e1ac29ec36/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:16.836206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256b695-9a8a-4251-93c8-2a73da1e5ff6 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:16.836502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256b695-9a8a-4251-93c8-2a73da1e5ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:16.836811 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256b695-9a8a-4251-93c8-2a73da1e5ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:16.854996 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-6256b695-9a8a-4251-93c8-2a73da1e5ff6 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:16.856175 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6256b695-9a8a-4251-93c8-2a73da1e5ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:16.918108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1101/2228] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:16.927908 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f574a02-2a35-48cd-99e2-20fd0d0e7839 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:16.929694 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-4f574a02-2a35-48cd-99e2-20fd0d0e7839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:16.989607 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f574a02-2a35-48cd-99e2-20fd0d0e7839 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:16.993371 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65a57a-88f3-4164-8248-a03dca50082d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:16.994012 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65a57a-88f3-4164-8248-a03dca50082d service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:16.994304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65a57a-88f3-4164-8248-a03dca50082d service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:16.994587 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65a57a-88f3-4164-8248-a03dca50082d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:17.003793 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65a57a-88f3-4164-8248-a03dca50082d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:17.005268 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65a57a-88f3-4164-8248-a03dca50082d service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:17.033308 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1128/2229] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:07:17.682877 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21470de1-3b60-4262-bbbe-7db13ad54d0e None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:17.683395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21470de1-3b60-4262-bbbe-7db13ad54d0e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:17.683506 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21470de1-3b60-4262-bbbe-7db13ad54d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:17.714369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-21470de1-3b60-4262-bbbe-7db13ad54d0e None None] MFA Rules not processed for user `4c137f9412de4914b77732e1ac29ec36`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:17.715740 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-21470de1-3b60-4262-bbbe-7db13ad54d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:17.790056 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1102/2230] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:17.801355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65a57a-88f3-4164-8248-a03dca50082d service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:17.804303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5e65a57a-88f3-4164-8248-a03dca50082d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:17.862082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65a57a-88f3-4164-8248-a03dca50082d service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:17.866336 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1772f85-0f4b-4fae-8993-a40751d3787a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:17.866983 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1772f85-0f4b-4fae-8993-a40751d3787a service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:17.867271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1772f85-0f4b-4fae-8993-a40751d3787a service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:17.867555 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1772f85-0f4b-4fae-8993-a40751d3787a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:17.883610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1772f85-0f4b-4fae-8993-a40751d3787a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:17.885152 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1772f85-0f4b-4fae-8993-a40751d3787a service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:17.895737 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1129/2231] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:07:19.268078 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b00ba-907e-4d8b-b6bd-0e555f0629a4 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:19.268654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b00ba-907e-4d8b-b6bd-0e555f0629a4 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:19.268654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b00ba-907e-4d8b-b6bd-0e555f0629a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:19.286083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-bb5b00ba-907e-4d8b-b6bd-0e555f0629a4 None None] MFA Rules not processed for user `4c137f9412de4914b77732e1ac29ec36`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:19.287300 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-bb5b00ba-907e-4d8b-b6bd-0e555f0629a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:19.296547 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1103/2232] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:19.672655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e715e-3361-492c-8d03-3250893114ef None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:19.673288 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e715e-3361-492c-8d03-3250893114ef None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:19.673469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e715e-3361-492c-8d03-3250893114ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:19.691248 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-074e715e-3361-492c-8d03-3250893114ef None None] MFA Rules not processed for user `64f97c33dcc74dd4ad69c277415f4643`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:19.692621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-074e715e-3361-492c-8d03-3250893114ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:19.706865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:19.760652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1130/2233] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:22.588214 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b00ba-907e-4d8b-b6bd-0e555f0629a4 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:22.589843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-bb5b00ba-907e-4d8b-b6bd-0e555f0629a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:22.614659 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b00ba-907e-4d8b-b6bd-0e555f0629a4 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:22.633616 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb90cf-46d6-41d5-8804-52b7ceaa4c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:22.635048 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb90cf-46d6-41d5-8804-52b7ceaa4c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:22.635501 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb90cf-46d6-41d5-8804-52b7ceaa4c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:22.635934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb90cf-46d6-41d5-8804-52b7ceaa4c4f admin admin] PATH_INFO: `/v3/users/1bef8cc467d34666bde95d1d830b1061` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:22.638539 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb90cf-46d6-41d5-8804-52b7ceaa4c4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bef8cc467d34666bde95d1d830b1061)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:22.641199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb90cf-46d6-41d5-8804-52b7ceaa4c4f admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:22.659644 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:22.722467 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-72eb90cf-46d6-41d5-8804-52b7ceaa4c4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bef8cc467d34666bde95d1d830b1061'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:22.737641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-72eb90cf-46d6-41d5-8804-52b7ceaa4c4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bef8cc467d34666bde95d1d830b1061'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:22.750602 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-72eb90cf-46d6-41d5-8804-52b7ceaa4c4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bef8cc467d34666bde95d1d830b1061'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:22.752173 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-72eb90cf-46d6-41d5-8804-52b7ceaa4c4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bef8cc467d34666bde95d1d830b1061'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:22.753320 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-72eb90cf-46d6-41d5-8804-52b7ceaa4c4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bef8cc467d34666bde95d1d830b1061'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:22.784798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1104/2234] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:07:22 2022] DELETE /identity/v3/users/1bef8cc467d34666bde95d1d830b1061 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:22.791137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6133e-cb37-4a71-9110-6d02c0063ba8 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:22.791722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6133e-cb37-4a71-9110-6d02c0063ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:22.791825 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6133e-cb37-4a71-9110-6d02c0063ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:22.812617 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-cec6133e-cb37-4a71-9110-6d02c0063ba8 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:22.813957 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-cec6133e-cb37-4a71-9110-6d02c0063ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:22.928148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1131/2235] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:07:22.936491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb90cf-46d6-41d5-8804-52b7ceaa4c4f admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:22.938714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-72eb90cf-46d6-41d5-8804-52b7ceaa4c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:23.065722 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb90cf-46d6-41d5-8804-52b7ceaa4c4f admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:23.071768 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d2e98-0cfa-443b-ba0a-716a3b540650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:23.072778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d2e98-0cfa-443b-ba0a-716a3b540650 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:23.073510 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d2e98-0cfa-443b-ba0a-716a3b540650 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:23.074271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d2e98-0cfa-443b-ba0a-716a3b540650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:23.087853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d2e98-0cfa-443b-ba0a-716a3b540650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:23.089803 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d2e98-0cfa-443b-ba0a-716a3b540650 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:23.106358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1105/2236] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:07:23.745809 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6133e-cb37-4a71-9110-6d02c0063ba8 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:23.747798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-cec6133e-cb37-4a71-9110-6d02c0063ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:23.762245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6133e-cb37-4a71-9110-6d02c0063ba8 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:23.808172 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae973a0-fd32-4135-9db2-a98045cd9955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:23.817982 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae973a0-fd32-4135-9db2-a98045cd9955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:23.818303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae973a0-fd32-4135-9db2-a98045cd9955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:23.818597 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae973a0-fd32-4135-9db2-a98045cd9955 admin admin] PATH_INFO: `/v3/projects/cc7dd17d21604b1e91c57cf999680743` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:23.825031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae973a0-fd32-4135-9db2-a98045cd9955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc7dd17d21604b1e91c57cf999680743)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:23.832411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae973a0-fd32-4135-9db2-a98045cd9955 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:23.908944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-7ae973a0-fd32-4135-9db2-a98045cd9955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc7dd17d21604b1e91c57cf999680743', 'request_id': 'req-7ae973a0-fd32-4135-9db2-a98045cd9955'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:23.917245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-7ae973a0-fd32-4135-9db2-a98045cd9955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc7dd17d21604b1e91c57cf999680743', 'request_id': 'req-7ae973a0-fd32-4135-9db2-a98045cd9955'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:23.927990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-7ae973a0-fd32-4135-9db2-a98045cd9955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc7dd17d21604b1e91c57cf999680743', 'request_id': 'req-7ae973a0-fd32-4135-9db2-a98045cd9955'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:23.943381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-7ae973a0-fd32-4135-9db2-a98045cd9955 admin admin] The token cache is being invalidate because project cc7dd17d21604b1e91c57cf999680743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:07:23.943685 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-7ae973a0-fd32-4135-9db2-a98045cd9955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:23.945876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-7ae973a0-fd32-4135-9db2-a98045cd9955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:23.948533 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1132/2237] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:07:23 2022] DELETE /identity/v3/projects/cc7dd17d21604b1e91c57cf999680743 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:23.953174 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588703e-f5f6-4e89-8270-3678b81a9daa None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:23.953695 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588703e-f5f6-4e89-8270-3678b81a9daa None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:23.953784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588703e-f5f6-4e89-8270-3678b81a9daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:23.996152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-2588703e-f5f6-4e89-8270-3678b81a9daa None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:23.997597 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-2588703e-f5f6-4e89-8270-3678b81a9daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:24.068306 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1106/2238] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:24.077532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e237d2-6509-4790-9a7f-613e43e5ba0a None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:24.077918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e237d2-6509-4790-9a7f-613e43e5ba0a None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:24.078288 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e237d2-6509-4790-9a7f-613e43e5ba0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:24.103079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-b0e237d2-6509-4790-9a7f-613e43e5ba0a None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:24.104314 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b0e237d2-6509-4790-9a7f-613e43e5ba0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:24.115206 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1133/2239] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:24.121266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588703e-f5f6-4e89-8270-3678b81a9daa None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:24.139355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588703e-f5f6-4e89-8270-3678b81a9daa None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:24.158399 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f7005-cdf2-46b6-8cd4-0f4cc0fc0950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:24.159106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f7005-cdf2-46b6-8cd4-0f4cc0fc0950 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:24.159398 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f7005-cdf2-46b6-8cd4-0f4cc0fc0950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:24.159679 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f7005-cdf2-46b6-8cd4-0f4cc0fc0950 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:24.160428 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f7005-cdf2-46b6-8cd4-0f4cc0fc0950 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:07:24.160789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f7005-cdf2-46b6-8cd4-0f4cc0fc0950 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:24.162597 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f7005-cdf2-46b6-8cd4-0f4cc0fc0950 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:24.182312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1107/2240] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:07:24.186703 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e237d2-6509-4790-9a7f-613e43e5ba0a None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:24.208993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e237d2-6509-4790-9a7f-613e43e5ba0a None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:24.220134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d3d77-7fc2-4629-84c0-d252fb340d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:24.221620 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d3d77-7fc2-4629-84c0-d252fb340d80 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:24.223154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d3d77-7fc2-4629-84c0-d252fb340d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:24.223514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d3d77-7fc2-4629-84c0-d252fb340d80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:24.226284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d3d77-7fc2-4629-84c0-d252fb340d80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:24.234026 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d3d77-7fc2-4629-84c0-d252fb340d80 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:24.306222 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1134/2241] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:07:24 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:07:24.307812 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f7005-cdf2-46b6-8cd4-0f4cc0fc0950 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:24.322223 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f7005-cdf2-46b6-8cd4-0f4cc0fc0950 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:24.343064 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:24.411577 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35beaa8c-80d4-4921-8516-614b69eb7344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:24.412938 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35beaa8c-80d4-4921-8516-614b69eb7344 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:24.413858 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35beaa8c-80d4-4921-8516-614b69eb7344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:24.415807 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35beaa8c-80d4-4921-8516-614b69eb7344 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:24.417643 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35beaa8c-80d4-4921-8516-614b69eb7344 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:24.420319 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35beaa8c-80d4-4921-8516-614b69eb7344 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:24.462296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1108/2242] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:07:24 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:07:24.464453 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d3d77-7fc2-4629-84c0-d252fb340d80 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:24.497137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d3d77-7fc2-4629-84c0-d252fb340d80 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:24.519831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2857d04-2e9c-4e13-b3e8-b00c6a702582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:24.520550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2857d04-2e9c-4e13-b3e8-b00c6a702582 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:24.521077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2857d04-2e9c-4e13-b3e8-b00c6a702582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:24.521364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2857d04-2e9c-4e13-b3e8-b00c6a702582 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:24.522220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2857d04-2e9c-4e13-b3e8-b00c6a702582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:24.524154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2857d04-2e9c-4e13-b3e8-b00c6a702582 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:24.536329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1135/2243] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:07:24.541526 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35beaa8c-80d4-4921-8516-614b69eb7344 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:24.553012 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35beaa8c-80d4-4921-8516-614b69eb7344 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:24.560676 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9fec7-3838-40e9-97c3-7924b5216b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:24.561474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9fec7-3838-40e9-97c3-7924b5216b2e admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:24.561819 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9fec7-3838-40e9-97c3-7924b5216b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:24.562201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9fec7-3838-40e9-97c3-7924b5216b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:24.563253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9fec7-3838-40e9-97c3-7924b5216b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:24.565035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9fec7-3838-40e9-97c3-7924b5216b2e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:24.577252 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1109/2244] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:07:24.581550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2857d04-2e9c-4e13-b3e8-b00c6a702582 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:24.594490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2857d04-2e9c-4e13-b3e8-b00c6a702582 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:24.599793 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c050979-0f44-43dd-a9e3-4134e7844d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:24.601654 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c050979-0f44-43dd-a9e3-4134e7844d57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:24.602016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c050979-0f44-43dd-a9e3-4134e7844d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:24.602880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c050979-0f44-43dd-a9e3-4134e7844d57 admin admin] PATH_INFO: `/v3/projects/6f2617282ba54313aca49e067a37e49d/users/682bec81ce6d48fda1cf80b3c9110166/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:24.605154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:24.626555 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c050979-0f44-43dd-a9e3-4134e7844d57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2617282ba54313aca49e067a37e49d, user_id=682bec81ce6d48fda1cf80b3c9110166, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:24.629594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c050979-0f44-43dd-a9e3-4134e7844d57 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:24.648285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1136/2245] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:07:24 2022] PUT /identity/v3/projects/6f2617282ba54313aca49e067a37e49d/users/682bec81ce6d48fda1cf80b3c9110166/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:07:24.727236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c8d0e-ac8d-43b2-896c-fe8ce6d7c06d None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:24.728229 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c8d0e-ac8d-43b2-896c-fe8ce6d7c06d None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:24.728647 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c8d0e-ac8d-43b2-896c-fe8ce6d7c06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:24.763943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-8b9c8d0e-ac8d-43b2-896c-fe8ce6d7c06d None None] MFA Rules not processed for user `682bec81ce6d48fda1cf80b3c9110166`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:24.765222 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8b9c8d0e-ac8d-43b2-896c-fe8ce6d7c06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:24.845307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1110/2246] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:24.856602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c050979-0f44-43dd-a9e3-4134e7844d57 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:24.858393 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2c050979-0f44-43dd-a9e3-4134e7844d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:24.923422 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c050979-0f44-43dd-a9e3-4134e7844d57 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:24.927228 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c3c75-f2a9-403c-8868-229a00950b81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:24.927951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c3c75-f2a9-403c-8868-229a00950b81 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:24.928962 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c3c75-f2a9-403c-8868-229a00950b81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:24.929314 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c3c75-f2a9-403c-8868-229a00950b81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:24.938860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c3c75-f2a9-403c-8868-229a00950b81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:24.940340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c3c75-f2a9-403c-8868-229a00950b81 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:24.953383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1137/2247] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:07:28.113054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c8d0e-ac8d-43b2-896c-fe8ce6d7c06d None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:28.117328 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8b9c8d0e-ac8d-43b2-896c-fe8ce6d7c06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:28.218892 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c3c75-f2a9-403c-8868-229a00950b81 service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:28.331243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c8d0e-ac8d-43b2-896c-fe8ce6d7c06d None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:28.336233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e1096-58a8-4fdd-8be1-ed02c97ffb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:28.337250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e1096-58a8-4fdd-8be1-ed02c97ffb7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:28.337606 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e1096-58a8-4fdd-8be1-ed02c97ffb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:28.338233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e1096-58a8-4fdd-8be1-ed02c97ffb7b admin admin] PATH_INFO: `/v3/users/0ee70760d01048adb24e92a12094d1fb` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:28.339270 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e1096-58a8-4fdd-8be1-ed02c97ffb7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ee70760d01048adb24e92a12094d1fb)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:28.341334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e1096-58a8-4fdd-8be1-ed02c97ffb7b admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:28.343893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c3c75-f2a9-403c-8868-229a00950b81 service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:28.353258 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da33fa-cf70-4bd7-9c47-a8e0bbd5503b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:28.354397 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da33fa-cf70-4bd7-9c47-a8e0bbd5503b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:28.355071 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da33fa-cf70-4bd7-9c47-a8e0bbd5503b admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:28.355071 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da33fa-cf70-4bd7-9c47-a8e0bbd5503b admin admin] PATH_INFO: `/v3/users/682bec81ce6d48fda1cf80b3c9110166` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:28.356400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da33fa-cf70-4bd7-9c47-a8e0bbd5503b admin admin] RBAC: Authorizing `identity:delete_user(user_id=682bec81ce6d48fda1cf80b3c9110166)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:28.358462 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da33fa-cf70-4bd7-9c47-a8e0bbd5503b admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:28.392493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:28.424394 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2a1e1096-58a8-4fdd-8be1-ed02c97ffb7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ee70760d01048adb24e92a12094d1fb'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:28.432222 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-68da33fa-cf70-4bd7-9c47-a8e0bbd5503b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '682bec81ce6d48fda1cf80b3c9110166'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:28.433321 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2a1e1096-58a8-4fdd-8be1-ed02c97ffb7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ee70760d01048adb24e92a12094d1fb'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:28.443228 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2a1e1096-58a8-4fdd-8be1-ed02c97ffb7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ee70760d01048adb24e92a12094d1fb'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:28.444884 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2a1e1096-58a8-4fdd-8be1-ed02c97ffb7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ee70760d01048adb24e92a12094d1fb'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:28.445922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2a1e1096-58a8-4fdd-8be1-ed02c97ffb7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ee70760d01048adb24e92a12094d1fb'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:28.451117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-68da33fa-cf70-4bd7-9c47-a8e0bbd5503b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '682bec81ce6d48fda1cf80b3c9110166'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:28.459115 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-68da33fa-cf70-4bd7-9c47-a8e0bbd5503b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '682bec81ce6d48fda1cf80b3c9110166'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:28.468951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-68da33fa-cf70-4bd7-9c47-a8e0bbd5503b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '682bec81ce6d48fda1cf80b3c9110166'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:28.469601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1111/2248] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:07:28 2022] DELETE /identity/v3/users/0ee70760d01048adb24e92a12094d1fb => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:28.473508 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-68da33fa-cf70-4bd7-9c47-a8e0bbd5503b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '682bec81ce6d48fda1cf80b3c9110166'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:28.484270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1138/2249] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:07:28 2022] DELETE /identity/v3/users/682bec81ce6d48fda1cf80b3c9110166 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:28.489130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f1c1d-ed25-47fd-aa8f-bac9fcfe98b5 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:28.489783 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f1c1d-ed25-47fd-aa8f-bac9fcfe98b5 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:28.489783 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f1c1d-ed25-47fd-aa8f-bac9fcfe98b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:28.509198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-4b4f1c1d-ed25-47fd-aa8f-bac9fcfe98b5 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:28.510804 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4b4f1c1d-ed25-47fd-aa8f-bac9fcfe98b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:28.592781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1112/2250] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:28.602938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da33fa-cf70-4bd7-9c47-a8e0bbd5503b admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:28.604340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-68da33fa-cf70-4bd7-9c47-a8e0bbd5503b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:28.611572 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:28.692687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da33fa-cf70-4bd7-9c47-a8e0bbd5503b admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:28.698666 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74850926-cddf-4140-a5c1-fa2d5fcd46e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:28.699404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74850926-cddf-4140-a5c1-fa2d5fcd46e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:28.699698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74850926-cddf-4140-a5c1-fa2d5fcd46e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:28.700002 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74850926-cddf-4140-a5c1-fa2d5fcd46e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:28.705480 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f1c1d-ed25-47fd-aa8f-bac9fcfe98b5 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:28.708361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4b4f1c1d-ed25-47fd-aa8f-bac9fcfe98b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:28.722367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74850926-cddf-4140-a5c1-fa2d5fcd46e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:28.724876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74850926-cddf-4140-a5c1-fa2d5fcd46e8 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:28.730019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f1c1d-ed25-47fd-aa8f-bac9fcfe98b5 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:28.736576 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f6d7a-c7a8-4482-b145-200ecd228ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:28.737807 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f6d7a-c7a8-4482-b145-200ecd228ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:28.738296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f6d7a-c7a8-4482-b145-200ecd228ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:28.738672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f6d7a-c7a8-4482-b145-200ecd228ec6 admin admin] PATH_INFO: `/v3/projects/7b528fe3a3be4b6eb49de69655952e36` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:28.739361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1139/2251] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:07:28.740934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f6d7a-c7a8-4482-b145-200ecd228ec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b528fe3a3be4b6eb49de69655952e36)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:28.745039 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f6d7a-c7a8-4482-b145-200ecd228ec6 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:28.817614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c35f6d7a-c7a8-4482-b145-200ecd228ec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b528fe3a3be4b6eb49de69655952e36', 'request_id': 'req-c35f6d7a-c7a8-4482-b145-200ecd228ec6'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:28.828642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c35f6d7a-c7a8-4482-b145-200ecd228ec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b528fe3a3be4b6eb49de69655952e36', 'request_id': 'req-c35f6d7a-c7a8-4482-b145-200ecd228ec6'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:28.839110 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c35f6d7a-c7a8-4482-b145-200ecd228ec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b528fe3a3be4b6eb49de69655952e36', 'request_id': 'req-c35f6d7a-c7a8-4482-b145-200ecd228ec6'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:28.850599 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c35f6d7a-c7a8-4482-b145-200ecd228ec6 admin admin] The token cache is being invalidate because project 7b528fe3a3be4b6eb49de69655952e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:07:28.851017 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c35f6d7a-c7a8-4482-b145-200ecd228ec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:28.851812 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c35f6d7a-c7a8-4482-b145-200ecd228ec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:28.853665 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1113/2252] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:07:28 2022] DELETE /identity/v3/projects/7b528fe3a3be4b6eb49de69655952e36 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:28.858218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74850926-cddf-4140-a5c1-fa2d5fcd46e8 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:28.869731 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74850926-cddf-4140-a5c1-fa2d5fcd46e8 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:28.942148 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f051bc-b88b-450c-b991-6b8cc6f4cf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:28.943109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f051bc-b88b-450c-b991-6b8cc6f4cf42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:28.943460 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f051bc-b88b-450c-b991-6b8cc6f4cf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:28.944832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f051bc-b88b-450c-b991-6b8cc6f4cf42 admin admin] PATH_INFO: `/v3/users/be0b66df14bd41c292e4058877d1019d` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:28.946203 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f051bc-b88b-450c-b991-6b8cc6f4cf42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be0b66df14bd41c292e4058877d1019d)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:28.948120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f051bc-b88b-450c-b991-6b8cc6f4cf42 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:29.013119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-20f051bc-b88b-450c-b991-6b8cc6f4cf42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be0b66df14bd41c292e4058877d1019d'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:29.026355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-20f051bc-b88b-450c-b991-6b8cc6f4cf42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be0b66df14bd41c292e4058877d1019d'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:29.033707 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-20f051bc-b88b-450c-b991-6b8cc6f4cf42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be0b66df14bd41c292e4058877d1019d'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:29.041835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-20f051bc-b88b-450c-b991-6b8cc6f4cf42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be0b66df14bd41c292e4058877d1019d'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:29.042638 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-20f051bc-b88b-450c-b991-6b8cc6f4cf42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be0b66df14bd41c292e4058877d1019d'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:29.048854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1140/2253] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:07:28 2022] DELETE /identity/v3/users/be0b66df14bd41c292e4058877d1019d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:07:29.142744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f6d7a-c7a8-4482-b145-200ecd228ec6 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:29.144628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c35f6d7a-c7a8-4482-b145-200ecd228ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:29.159222 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f6d7a-c7a8-4482-b145-200ecd228ec6 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:29.197325 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:29.238239 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c00e57-8b9c-4efe-a137-9e54b9631627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:29.239029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c00e57-8b9c-4efe-a137-9e54b9631627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:29.239325 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c00e57-8b9c-4efe-a137-9e54b9631627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:29.239609 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c00e57-8b9c-4efe-a137-9e54b9631627 admin admin] PATH_INFO: `/v3/projects/6f2617282ba54313aca49e067a37e49d` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:29.242277 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c00e57-8b9c-4efe-a137-9e54b9631627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f2617282ba54313aca49e067a37e49d)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:29.243885 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c00e57-8b9c-4efe-a137-9e54b9631627 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:29.288238 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f051bc-b88b-450c-b991-6b8cc6f4cf42 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:29.291571 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-20f051bc-b88b-450c-b991-6b8cc6f4cf42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:29.310967 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-02c00e57-8b9c-4efe-a137-9e54b9631627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f2617282ba54313aca49e067a37e49d', 'request_id': 'req-02c00e57-8b9c-4efe-a137-9e54b9631627'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:29.317532 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-02c00e57-8b9c-4efe-a137-9e54b9631627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f2617282ba54313aca49e067a37e49d', 'request_id': 'req-02c00e57-8b9c-4efe-a137-9e54b9631627'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:29.323636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f051bc-b88b-450c-b991-6b8cc6f4cf42 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:29.327403 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-02c00e57-8b9c-4efe-a137-9e54b9631627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f2617282ba54313aca49e067a37e49d', 'request_id': 'req-02c00e57-8b9c-4efe-a137-9e54b9631627'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:29.356486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-02c00e57-8b9c-4efe-a137-9e54b9631627 admin admin] The token cache is being invalidate because project 6f2617282ba54313aca49e067a37e49d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:07:29.356889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-02c00e57-8b9c-4efe-a137-9e54b9631627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:29.357753 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-02c00e57-8b9c-4efe-a137-9e54b9631627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:29.360768 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1114/2254] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:07:29 2022] DELETE /identity/v3/projects/6f2617282ba54313aca49e067a37e49d => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:07:29.392130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:29.450543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b662f1c-2a18-45a3-8187-727e5a8e149e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:29.451445 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b662f1c-2a18-45a3-8187-727e5a8e149e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:29.451727 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b662f1c-2a18-45a3-8187-727e5a8e149e admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:29.452004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b662f1c-2a18-45a3-8187-727e5a8e149e admin admin] PATH_INFO: `/v3/projects/264ee15846074e94b675502ac4ac025a` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:29.456481 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b662f1c-2a18-45a3-8187-727e5a8e149e admin admin] RBAC: Authorizing `identity:delete_project(project_id=264ee15846074e94b675502ac4ac025a)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:29.459942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b662f1c-2a18-45a3-8187-727e5a8e149e admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:29.623397 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3b662f1c-2a18-45a3-8187-727e5a8e149e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '264ee15846074e94b675502ac4ac025a', 'request_id': 'req-3b662f1c-2a18-45a3-8187-727e5a8e149e'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:29.630913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3b662f1c-2a18-45a3-8187-727e5a8e149e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '264ee15846074e94b675502ac4ac025a', 'request_id': 'req-3b662f1c-2a18-45a3-8187-727e5a8e149e'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:29.643803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3b662f1c-2a18-45a3-8187-727e5a8e149e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '264ee15846074e94b675502ac4ac025a', 'request_id': 'req-3b662f1c-2a18-45a3-8187-727e5a8e149e'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:29.662008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3b662f1c-2a18-45a3-8187-727e5a8e149e admin admin] The token cache is being invalidate because project 264ee15846074e94b675502ac4ac025a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:07:29.663508 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3b662f1c-2a18-45a3-8187-727e5a8e149e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:29.664597 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3b662f1c-2a18-45a3-8187-727e5a8e149e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:29.666997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1141/2255] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:07:29 2022] DELETE /identity/v3/projects/264ee15846074e94b675502ac4ac025a => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:29.672278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a769d-990b-4814-bae1-712af73addbb None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:29.673024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a769d-990b-4814-bae1-712af73addbb None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:29.673623 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a769d-990b-4814-bae1-712af73addbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:29.708478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-5e8a769d-990b-4814-bae1-712af73addbb None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:29.708478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-5e8a769d-990b-4814-bae1-712af73addbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:29.780190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1115/2256] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:29.789584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e6d03-22cd-4d04-9aca-2ac70e04e180 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:29.789884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e6d03-22cd-4d04-9aca-2ac70e04e180 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:29.790220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e6d03-22cd-4d04-9aca-2ac70e04e180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:29.810543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-5a3e6d03-22cd-4d04-9aca-2ac70e04e180 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:29.811792 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5a3e6d03-22cd-4d04-9aca-2ac70e04e180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:29.823391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1142/2257] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:07:29.828071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a769d-990b-4814-bae1-712af73addbb None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:29.841273 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a769d-990b-4814-bae1-712af73addbb None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:29.847494 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6625f-7a9f-46f8-b9fe-c032c5bd317e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:29.848280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6625f-7a9f-46f8-b9fe-c032c5bd317e admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:29.848681 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6625f-7a9f-46f8-b9fe-c032c5bd317e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:29.849218 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6625f-7a9f-46f8-b9fe-c032c5bd317e admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:29.850132 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6625f-7a9f-46f8-b9fe-c032c5bd317e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:07:29.850566 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6625f-7a9f-46f8-b9fe-c032c5bd317e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:29.852621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6625f-7a9f-46f8-b9fe-c032c5bd317e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:29.868103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1116/2258] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 22:07:29.874854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e6d03-22cd-4d04-9aca-2ac70e04e180 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:29.891526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e6d03-22cd-4d04-9aca-2ac70e04e180 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:29.900632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ba5e6-6ff7-414e-ab4c-929551536bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:29.901448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ba5e6-6ff7-414e-ab4c-929551536bdf admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:29.901794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ba5e6-6ff7-414e-ab4c-929551536bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:29.902141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ba5e6-6ff7-414e-ab4c-929551536bdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:29.903226 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ba5e6-6ff7-414e-ab4c-929551536bdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:29.905218 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ba5e6-6ff7-414e-ab4c-929551536bdf admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:29.945850 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1143/2259] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:07:29 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:07:29.947239 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6625f-7a9f-46f8-b9fe-c032c5bd317e admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:29.958677 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6625f-7a9f-46f8-b9fe-c032c5bd317e admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:30.004191 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:30.019836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de9722-f747-432b-8ddc-511fa70e88b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:30.020497 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de9722-f747-432b-8ddc-511fa70e88b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:30.020822 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de9722-f747-432b-8ddc-511fa70e88b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:30.021102 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de9722-f747-432b-8ddc-511fa70e88b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:30.022757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de9722-f747-432b-8ddc-511fa70e88b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:30.024763 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de9722-f747-432b-8ddc-511fa70e88b0 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:30.044770 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1117/2260] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:07:29 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:07:30.049303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ba5e6-6ff7-414e-ab4c-929551536bdf admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:30.059782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ba5e6-6ff7-414e-ab4c-929551536bdf admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:30.065091 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e04cd-ced1-438e-bbfd-b71e83903516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:30.065819 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e04cd-ced1-438e-bbfd-b71e83903516 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:30.066111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e04cd-ced1-438e-bbfd-b71e83903516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:30.066429 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e04cd-ced1-438e-bbfd-b71e83903516 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:30.067238 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e04cd-ced1-438e-bbfd-b71e83903516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:30.070166 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e04cd-ced1-438e-bbfd-b71e83903516 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:30.081624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1144/2261] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:07:30.086587 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de9722-f747-432b-8ddc-511fa70e88b0 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:30.097341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de9722-f747-432b-8ddc-511fa70e88b0 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:30.102821 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b72ca5-cffe-45ec-8055-ae22c32ed78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:30.103508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b72ca5-cffe-45ec-8055-ae22c32ed78e admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:30.103914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b72ca5-cffe-45ec-8055-ae22c32ed78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:30.104143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b72ca5-cffe-45ec-8055-ae22c32ed78e admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:30.104951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b72ca5-cffe-45ec-8055-ae22c32ed78e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:30.106543 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b72ca5-cffe-45ec-8055-ae22c32ed78e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:30.117270 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1118/2262] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:07:30.121445 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e04cd-ced1-438e-bbfd-b71e83903516 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:30.133358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e04cd-ced1-438e-bbfd-b71e83903516 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:30.138589 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16d7ba-3376-4e6d-9a41-76d26f69621e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:30.139570 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16d7ba-3376-4e6d-9a41-76d26f69621e admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:30.139896 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16d7ba-3376-4e6d-9a41-76d26f69621e admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:30.140213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16d7ba-3376-4e6d-9a41-76d26f69621e admin admin] PATH_INFO: `/v3/projects/84e5d658058a479ab5984ddf506aef40/users/ee16a2051be840c7bcbd108c3b204058/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:30.156657 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16d7ba-3376-4e6d-9a41-76d26f69621e admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e5d658058a479ab5984ddf506aef40, user_id=ee16a2051be840c7bcbd108c3b204058, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:30.159619 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16d7ba-3376-4e6d-9a41-76d26f69621e admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:30.174161 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1145/2263] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:07:30 2022] PUT /identity/v3/projects/84e5d658058a479ab5984ddf506aef40/users/ee16a2051be840c7bcbd108c3b204058/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:30.185298 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ffb72-a4dd-4518-8466-0fea0e4cad14 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:30.185819 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ffb72-a4dd-4518-8466-0fea0e4cad14 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:30.185913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ffb72-a4dd-4518-8466-0fea0e4cad14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:30.216568 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-223ffb72-a4dd-4518-8466-0fea0e4cad14 None None] MFA Rules not processed for user `ee16a2051be840c7bcbd108c3b204058`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:30.217863 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-223ffb72-a4dd-4518-8466-0fea0e4cad14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:30.293736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1119/2264] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:30.306560 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16d7ba-3376-4e6d-9a41-76d26f69621e admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:30.308339 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2b16d7ba-3376-4e6d-9a41-76d26f69621e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:30.370777 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16d7ba-3376-4e6d-9a41-76d26f69621e admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:30.374997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809ca11-9b2c-44e1-9598-5823b9fcb7d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:30.375648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809ca11-9b2c-44e1-9598-5823b9fcb7d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:30.375931 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809ca11-9b2c-44e1-9598-5823b9fcb7d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:30.376208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809ca11-9b2c-44e1-9598-5823b9fcb7d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:30.383526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:30.386358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809ca11-9b2c-44e1-9598-5823b9fcb7d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:30.388249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809ca11-9b2c-44e1-9598-5823b9fcb7d3 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:30.400980 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1146/2265] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:07:30.419859 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ffb72-a4dd-4518-8466-0fea0e4cad14 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:30.487726 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ffb72-a4dd-4518-8466-0fea0e4cad14 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:30.495499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a1a5b-5aad-41e8-9a72-be9a2a9e0759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:30.496667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a1a5b-5aad-41e8-9a72-be9a2a9e0759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:30.496667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a1a5b-5aad-41e8-9a72-be9a2a9e0759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:30.497491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a1a5b-5aad-41e8-9a72-be9a2a9e0759 admin admin] PATH_INFO: `/v3/users/ee16a2051be840c7bcbd108c3b204058` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:30.498592 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a1a5b-5aad-41e8-9a72-be9a2a9e0759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee16a2051be840c7bcbd108c3b204058)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:30.500573 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a1a5b-5aad-41e8-9a72-be9a2a9e0759 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:30.566021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-de7a1a5b-5aad-41e8-9a72-be9a2a9e0759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee16a2051be840c7bcbd108c3b204058'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:30.579394 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-de7a1a5b-5aad-41e8-9a72-be9a2a9e0759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee16a2051be840c7bcbd108c3b204058'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:30.581974 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:30.593828 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-de7a1a5b-5aad-41e8-9a72-be9a2a9e0759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee16a2051be840c7bcbd108c3b204058'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:30.595193 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-de7a1a5b-5aad-41e8-9a72-be9a2a9e0759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee16a2051be840c7bcbd108c3b204058'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:30.596730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-de7a1a5b-5aad-41e8-9a72-be9a2a9e0759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee16a2051be840c7bcbd108c3b204058'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:30.629363 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1120/2266] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:07:30 2022] DELETE /identity/v3/users/ee16a2051be840c7bcbd108c3b204058 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:30.634455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4cc52-8cc8-4913-b412-92ea239e4687 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:30.634843 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4cc52-8cc8-4913-b412-92ea239e4687 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:30.635202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4cc52-8cc8-4913-b412-92ea239e4687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:30.655319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-5fb4cc52-8cc8-4913-b412-92ea239e4687 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:30.657031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5fb4cc52-8cc8-4913-b412-92ea239e4687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:30.762356 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a1a5b-5aad-41e8-9a72-be9a2a9e0759 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:30.763835 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-de7a1a5b-5aad-41e8-9a72-be9a2a9e0759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:30.764843 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1147/2267] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:30.850619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a1a5b-5aad-41e8-9a72-be9a2a9e0759 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:30.856236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bca95-8e1d-461a-81c9-40dfa859c675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:30.857016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bca95-8e1d-461a-81c9-40dfa859c675 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:30.857398 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bca95-8e1d-461a-81c9-40dfa859c675 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:30.857780 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bca95-8e1d-461a-81c9-40dfa859c675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:30.870214 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bca95-8e1d-461a-81c9-40dfa859c675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:30.872177 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bca95-8e1d-461a-81c9-40dfa859c675 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:30.888320 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1121/2268] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:07:31.436568 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4cc52-8cc8-4913-b412-92ea239e4687 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:31.438099 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5fb4cc52-8cc8-4913-b412-92ea239e4687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:31.453305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4cc52-8cc8-4913-b412-92ea239e4687 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:31.474991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42173f-957c-4cec-84ae-319eb7d03ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:31.476587 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42173f-957c-4cec-84ae-319eb7d03ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:31.478275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42173f-957c-4cec-84ae-319eb7d03ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:31.478782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42173f-957c-4cec-84ae-319eb7d03ce3 admin admin] PATH_INFO: `/v3/projects/84e5d658058a479ab5984ddf506aef40` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:31.483317 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42173f-957c-4cec-84ae-319eb7d03ce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84e5d658058a479ab5984ddf506aef40)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:31.485922 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42173f-957c-4cec-84ae-319eb7d03ce3 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:31.538224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-8d42173f-957c-4cec-84ae-319eb7d03ce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84e5d658058a479ab5984ddf506aef40', 'request_id': 'req-8d42173f-957c-4cec-84ae-319eb7d03ce3'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:31.543591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-8d42173f-957c-4cec-84ae-319eb7d03ce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84e5d658058a479ab5984ddf506aef40', 'request_id': 'req-8d42173f-957c-4cec-84ae-319eb7d03ce3'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:31.550573 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-8d42173f-957c-4cec-84ae-319eb7d03ce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84e5d658058a479ab5984ddf506aef40', 'request_id': 'req-8d42173f-957c-4cec-84ae-319eb7d03ce3'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:31.562556 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-8d42173f-957c-4cec-84ae-319eb7d03ce3 admin admin] The token cache is being invalidate because project 84e5d658058a479ab5984ddf506aef40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:07:31.562844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-8d42173f-957c-4cec-84ae-319eb7d03ce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:31.563502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-8d42173f-957c-4cec-84ae-319eb7d03ce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:31.565096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1148/2269] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:07:31 2022] DELETE /identity/v3/projects/84e5d658058a479ab5984ddf506aef40 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:31.570892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988b262-3565-40d3-ac39-010bc6184aea None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:31.571391 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988b262-3565-40d3-ac39-010bc6184aea None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:31.571483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988b262-3565-40d3-ac39-010bc6184aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:31.590470 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-8988b262-3565-40d3-ac39-010bc6184aea None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:31.591595 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8988b262-3565-40d3-ac39-010bc6184aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:31.651452 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1122/2270] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:07:31.659252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f7df5-097e-4d19-acd3-95fc9ff49043 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:31.659547 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f7df5-097e-4d19-acd3-95fc9ff49043 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:31.659829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f7df5-097e-4d19-acd3-95fc9ff49043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:31.679337 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-243f7df5-097e-4d19-acd3-95fc9ff49043 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:31.680552 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-243f7df5-097e-4d19-acd3-95fc9ff49043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:31.688561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1149/2271] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:31.692936 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988b262-3565-40d3-ac39-010bc6184aea None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:31.703785 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988b262-3565-40d3-ac39-010bc6184aea None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:31.708381 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979bc6a-af1f-41b2-bf98-4f996bfe7351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:31.709130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979bc6a-af1f-41b2-bf98-4f996bfe7351 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:31.709427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979bc6a-af1f-41b2-bf98-4f996bfe7351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:31.709765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979bc6a-af1f-41b2-bf98-4f996bfe7351 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:31.710457 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e979bc6a-af1f-41b2-bf98-4f996bfe7351 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:07:31.710753 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e979bc6a-af1f-41b2-bf98-4f996bfe7351 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:31.712358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e979bc6a-af1f-41b2-bf98-4f996bfe7351 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:31.725142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1123/2272] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:07:31.729790 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f7df5-097e-4d19-acd3-95fc9ff49043 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:31.741031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f7df5-097e-4d19-acd3-95fc9ff49043 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:31.745916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b3a7f-f914-48ce-8595-8f422bd9edc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:31.746594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b3a7f-f914-48ce-8595-8f422bd9edc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:31.746879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b3a7f-f914-48ce-8595-8f422bd9edc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:31.747162 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b3a7f-f914-48ce-8595-8f422bd9edc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:31.747870 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b3a7f-f914-48ce-8595-8f422bd9edc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:31.749864 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b3a7f-f914-48ce-8595-8f422bd9edc9 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:31.780488 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1150/2273] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:07:31 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:07:31.785196 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979bc6a-af1f-41b2-bf98-4f996bfe7351 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:31.798798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979bc6a-af1f-41b2-bf98-4f996bfe7351 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:31.887196 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d73c8a-33a2-416d-82d1-6cf9ea1aa09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:31.887887 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d73c8a-33a2-416d-82d1-6cf9ea1aa09e admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:31.888194 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d73c8a-33a2-416d-82d1-6cf9ea1aa09e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:31.888458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d73c8a-33a2-416d-82d1-6cf9ea1aa09e admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:31.889199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d73c8a-33a2-416d-82d1-6cf9ea1aa09e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:31.891157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d73c8a-33a2-416d-82d1-6cf9ea1aa09e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:31.901124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:31.924659 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1124/2274] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:07:31 2022] POST /identity/v3/users => generated 292 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:07:31.925785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b3a7f-f914-48ce-8595-8f422bd9edc9 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:31.940654 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b3a7f-f914-48ce-8595-8f422bd9edc9 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:31.948136 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4bf2e8-ec7c-4007-89b6-e245df8aa06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:31.948854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bf2e8-ec7c-4007-89b6-e245df8aa06a admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:31.949181 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bf2e8-ec7c-4007-89b6-e245df8aa06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:31.949461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bf2e8-ec7c-4007-89b6-e245df8aa06a admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:31.950203 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4bf2e8-ec7c-4007-89b6-e245df8aa06a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:31.951855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4bf2e8-ec7c-4007-89b6-e245df8aa06a admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:31.966313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1151/2275] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:07:31.969951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d73c8a-33a2-416d-82d1-6cf9ea1aa09e admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:31.987714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d73c8a-33a2-416d-82d1-6cf9ea1aa09e admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:31.994104 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9ee48-6832-4448-b9ba-0e8ca2b33260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:31.994915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9ee48-6832-4448-b9ba-0e8ca2b33260 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:31.995209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9ee48-6832-4448-b9ba-0e8ca2b33260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:31.995567 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9ee48-6832-4448-b9ba-0e8ca2b33260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:31.996331 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9ee48-6832-4448-b9ba-0e8ca2b33260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:31.997994 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9ee48-6832-4448-b9ba-0e8ca2b33260 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:32.012256 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1125/2276] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:07:32.017308 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4bf2e8-ec7c-4007-89b6-e245df8aa06a admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:32.031004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4bf2e8-ec7c-4007-89b6-e245df8aa06a admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:32.055217 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d0d8e-b3ad-4948-a235-93d0a4550bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:32.056138 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d0d8e-b3ad-4948-a235-93d0a4550bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:32.056445 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d0d8e-b3ad-4948-a235-93d0a4550bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:32.056921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d0d8e-b3ad-4948-a235-93d0a4550bc4 admin admin] PATH_INFO: `/v3/projects/51584323f85e49ab98108553b7cd353c/users/830fba9768f6484badeb8caa14d6ce51/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:32.086863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d0d8e-b3ad-4948-a235-93d0a4550bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51584323f85e49ab98108553b7cd353c, user_id=830fba9768f6484badeb8caa14d6ce51, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:32.089697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d0d8e-b3ad-4948-a235-93d0a4550bc4 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:32.110025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1152/2277] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:07:32 2022] PUT /identity/v3/projects/51584323f85e49ab98108553b7cd353c/users/830fba9768f6484badeb8caa14d6ce51/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:32.124266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82349984-74a7-46e3-bf77-5b9fdbda23da None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:32.124580 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82349984-74a7-46e3-bf77-5b9fdbda23da None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:32.124995 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82349984-74a7-46e3-bf77-5b9fdbda23da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:32.173458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-82349984-74a7-46e3-bf77-5b9fdbda23da None None] MFA Rules not processed for user `830fba9768f6484badeb8caa14d6ce51`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:32.174767 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-82349984-74a7-46e3-bf77-5b9fdbda23da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:32.281278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1126/2278] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:32.289630 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d0d8e-b3ad-4948-a235-93d0a4550bc4 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:32.315347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:32.372195 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d0d8e-b3ad-4948-a235-93d0a4550bc4 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:32.377250 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784f2d0-b368-44ae-9de3-4d8cb9be79c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:32.377905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784f2d0-b368-44ae-9de3-4d8cb9be79c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:32.378191 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784f2d0-b368-44ae-9de3-4d8cb9be79c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:32.378510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784f2d0-b368-44ae-9de3-4d8cb9be79c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:32.379261 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784f2d0-b368-44ae-9de3-4d8cb9be79c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:32.381219 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784f2d0-b368-44ae-9de3-4d8cb9be79c1 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:32.407948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1153/2279] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:07:32 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:07:32.412069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82349984-74a7-46e3-bf77-5b9fdbda23da None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:32.423740 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82349984-74a7-46e3-bf77-5b9fdbda23da None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:32.486705 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a967c-edb1-4292-9e5a-8d886889c620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:32.487784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a967c-edb1-4292-9e5a-8d886889c620 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:32.488140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a967c-edb1-4292-9e5a-8d886889c620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:32.488615 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a967c-edb1-4292-9e5a-8d886889c620 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:32.489461 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a967c-edb1-4292-9e5a-8d886889c620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:32.491911 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a967c-edb1-4292-9e5a-8d886889c620 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:32.514471 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1127/2280] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:07:32 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:07:32.518887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784f2d0-b368-44ae-9de3-4d8cb9be79c1 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:32.530629 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784f2d0-b368-44ae-9de3-4d8cb9be79c1 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:32.536183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f517d-2b77-4646-92ca-edaa90b701f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:32.536887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f517d-2b77-4646-92ca-edaa90b701f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:32.537781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f517d-2b77-4646-92ca-edaa90b701f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:32.538077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f517d-2b77-4646-92ca-edaa90b701f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:32.538816 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f517d-2b77-4646-92ca-edaa90b701f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:32.540431 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f517d-2b77-4646-92ca-edaa90b701f6 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:32.554961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1154/2281] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:07:32.556361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a967c-edb1-4292-9e5a-8d886889c620 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:32.567088 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a967c-edb1-4292-9e5a-8d886889c620 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:32.572501 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5855b5-0cbe-4031-87ba-636f69ca5fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:32.573878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5855b5-0cbe-4031-87ba-636f69ca5fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:32.574348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5855b5-0cbe-4031-87ba-636f69ca5fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:32.574640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5855b5-0cbe-4031-87ba-636f69ca5fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:32.575382 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5855b5-0cbe-4031-87ba-636f69ca5fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:32.577049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5855b5-0cbe-4031-87ba-636f69ca5fe5 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:32.591902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1128/2282] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:07:32.593271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f517d-2b77-4646-92ca-edaa90b701f6 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:32.611233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f517d-2b77-4646-92ca-edaa90b701f6 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:32.617480 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f9ab1-cfe3-4ce5-9fb6-aa8d77e0a8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:32.618442 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f9ab1-cfe3-4ce5-9fb6-aa8d77e0a8e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:32.618773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f9ab1-cfe3-4ce5-9fb6-aa8d77e0a8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:32.619198 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f9ab1-cfe3-4ce5-9fb6-aa8d77e0a8e4 admin admin] PATH_INFO: `/v3/projects/2566f0119f394602a92602df9532cadf/users/eff3cdaa69f34fcfb7f8198ebcb8faa4/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:32.639291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f9ab1-cfe3-4ce5-9fb6-aa8d77e0a8e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2566f0119f394602a92602df9532cadf, user_id=eff3cdaa69f34fcfb7f8198ebcb8faa4, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:32.642400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f9ab1-cfe3-4ce5-9fb6-aa8d77e0a8e4 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:32.659590 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1155/2283] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:07:32 2022] PUT /identity/v3/projects/2566f0119f394602a92602df9532cadf/users/eff3cdaa69f34fcfb7f8198ebcb8faa4/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:07:32.677203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fc70b-1e60-4c5f-9142-602cdd7c3fec None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:32.677748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fc70b-1e60-4c5f-9142-602cdd7c3fec None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:32.678372 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fc70b-1e60-4c5f-9142-602cdd7c3fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:32.713404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-8a9fc70b-1e60-4c5f-9142-602cdd7c3fec None None] MFA Rules not processed for user `eff3cdaa69f34fcfb7f8198ebcb8faa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:32.714915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8a9fc70b-1e60-4c5f-9142-602cdd7c3fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:32.762708 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:32.792953 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1129/2284] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:32.798178 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f9ab1-cfe3-4ce5-9fb6-aa8d77e0a8e4 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:32.868687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f9ab1-cfe3-4ce5-9fb6-aa8d77e0a8e4 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:32.874180 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb90719-8bed-41fd-8bfb-8ca70103d868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:32.874913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb90719-8bed-41fd-8bfb-8ca70103d868 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:32.875307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb90719-8bed-41fd-8bfb-8ca70103d868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:32.875589 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb90719-8bed-41fd-8bfb-8ca70103d868 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:32.876371 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb90719-8bed-41fd-8bfb-8ca70103d868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:32.878821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb90719-8bed-41fd-8bfb-8ca70103d868 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:32.909960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1156/2285] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:07:32 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:07:32.914064 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fc70b-1e60-4c5f-9142-602cdd7c3fec None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:32.925063 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fc70b-1e60-4c5f-9142-602cdd7c3fec None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:32.993867 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834deab-6f24-402b-864c-03b6f3b8d744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:32.994531 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834deab-6f24-402b-864c-03b6f3b8d744 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:32.994814 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834deab-6f24-402b-864c-03b6f3b8d744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:32.995089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834deab-6f24-402b-864c-03b6f3b8d744 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:32.995778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b834deab-6f24-402b-864c-03b6f3b8d744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:32.997724 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b834deab-6f24-402b-864c-03b6f3b8d744 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:33.033929 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1130/2286] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:07:32 2022] POST /identity/v3/users => generated 298 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:07:33.038386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb90719-8bed-41fd-8bfb-8ca70103d868 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:33.050341 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb90719-8bed-41fd-8bfb-8ca70103d868 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:33.055638 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861253b0-964d-4eee-87e7-f4e8ffc128de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:33.056395 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861253b0-964d-4eee-87e7-f4e8ffc128de admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:33.056781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861253b0-964d-4eee-87e7-f4e8ffc128de admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:33.057130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861253b0-964d-4eee-87e7-f4e8ffc128de admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:33.057930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861253b0-964d-4eee-87e7-f4e8ffc128de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:33.059602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861253b0-964d-4eee-87e7-f4e8ffc128de admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:33.074133 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1157/2287] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:07:33.078434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834deab-6f24-402b-864c-03b6f3b8d744 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:33.089173 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834deab-6f24-402b-864c-03b6f3b8d744 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:33.094291 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1337828-f1e7-44cb-82f6-291c0208404c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:33.095278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1337828-f1e7-44cb-82f6-291c0208404c admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:33.095626 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1337828-f1e7-44cb-82f6-291c0208404c admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:33.095970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1337828-f1e7-44cb-82f6-291c0208404c admin admin] PATH_INFO: `/v3/projects/e4f716ec22b24a128884fde16e969a07/users/eee2d98b64d44b14a58d3bac3a507283/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:33.112996 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1337828-f1e7-44cb-82f6-291c0208404c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4f716ec22b24a128884fde16e969a07, user_id=eee2d98b64d44b14a58d3bac3a507283, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:33.118582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1337828-f1e7-44cb-82f6-291c0208404c admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:33.131871 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1131/2288] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:07:33 2022] PUT /identity/v3/projects/e4f716ec22b24a128884fde16e969a07/users/eee2d98b64d44b14a58d3bac3a507283/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:33.135740 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861253b0-964d-4eee-87e7-f4e8ffc128de admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:33.146236 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861253b0-964d-4eee-87e7-f4e8ffc128de admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:33.187185 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:33.208836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763e586-fc2b-4a45-bc07-56dc6c1494cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:33.210300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763e586-fc2b-4a45-bc07-56dc6c1494cf admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:33.210664 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763e586-fc2b-4a45-bc07-56dc6c1494cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:33.210952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763e586-fc2b-4a45-bc07-56dc6c1494cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:33.211687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763e586-fc2b-4a45-bc07-56dc6c1494cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:33.213478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763e586-fc2b-4a45-bc07-56dc6c1494cf admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:33.224044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1158/2289] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:07:33.230100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1337828-f1e7-44cb-82f6-291c0208404c admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:33.244366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1337828-f1e7-44cb-82f6-291c0208404c admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:33.249759 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ee967-a138-447e-9aea-555f63fd2fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:33.250465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ee967-a138-447e-9aea-555f63fd2fc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:33.251307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ee967-a138-447e-9aea-555f63fd2fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:33.251599 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ee967-a138-447e-9aea-555f63fd2fc6 admin admin] PATH_INFO: `/v3/projects/e4f716ec22b24a128884fde16e969a07/users/eee2d98b64d44b14a58d3bac3a507283/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:33.253872 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ee967-a138-447e-9aea-555f63fd2fc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4f716ec22b24a128884fde16e969a07, user_id=eee2d98b64d44b14a58d3bac3a507283, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:33.257203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ee967-a138-447e-9aea-555f63fd2fc6 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:33.278330 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1132/2290] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:07:33 2022] PUT /identity/v3/projects/e4f716ec22b24a128884fde16e969a07/users/eee2d98b64d44b14a58d3bac3a507283/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:33.289469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18baae-3bbe-4af9-aa86-7d47dd622039 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:33.289975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18baae-3bbe-4af9-aa86-7d47dd622039 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:33.290452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18baae-3bbe-4af9-aa86-7d47dd622039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:33.348410 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-ee18baae-3bbe-4af9-aa86-7d47dd622039 None None] MFA Rules not processed for user `eee2d98b64d44b14a58d3bac3a507283`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:33.349869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ee18baae-3bbe-4af9-aa86-7d47dd622039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:33.443497 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1159/2291] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:33.450922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ee967-a138-447e-9aea-555f63fd2fc6 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:33.452999 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-121ee967-a138-447e-9aea-555f63fd2fc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:33.529252 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ee967-a138-447e-9aea-555f63fd2fc6 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:33.533748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d721119-770b-481e-a55b-a4423a7ae743 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:33.534403 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d721119-770b-481e-a55b-a4423a7ae743 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:33.534692 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d721119-770b-481e-a55b-a4423a7ae743 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:33.535002 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d721119-770b-481e-a55b-a4423a7ae743 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:33.584361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:33.599222 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d721119-770b-481e-a55b-a4423a7ae743 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:33.601577 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d721119-770b-481e-a55b-a4423a7ae743 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:33.612129 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1133/2292] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:07:35.231672 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18baae-3bbe-4af9-aa86-7d47dd622039 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:35.233234 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ee18baae-3bbe-4af9-aa86-7d47dd622039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:35.246088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18baae-3bbe-4af9-aa86-7d47dd622039 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:35.249783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc0708-a87c-47b2-b6e4-98d53301e3db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:35.250402 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc0708-a87c-47b2-b6e4-98d53301e3db service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:35.250686 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc0708-a87c-47b2-b6e4-98d53301e3db service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:35.250961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc0708-a87c-47b2-b6e4-98d53301e3db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:35.310990 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc0708-a87c-47b2-b6e4-98d53301e3db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:35.312569 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc0708-a87c-47b2-b6e4-98d53301e3db service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:35.322764 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1160/2293] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:07:35.926863 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d721119-770b-481e-a55b-a4423a7ae743 service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:35.937364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d721119-770b-481e-a55b-a4423a7ae743 service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:35.941198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee473c9-2468-469f-b1da-e12364bd9ba8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:35.941876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee473c9-2468-469f-b1da-e12364bd9ba8 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:35.942264 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee473c9-2468-469f-b1da-e12364bd9ba8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:35.942593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee473c9-2468-469f-b1da-e12364bd9ba8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:35.952978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee473c9-2468-469f-b1da-e12364bd9ba8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:35.954450 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee473c9-2468-469f-b1da-e12364bd9ba8 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:35.968478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1134/2294] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:07:39.951843 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc0708-a87c-47b2-b6e4-98d53301e3db service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:40.017558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc0708-a87c-47b2-b6e4-98d53301e3db service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:40.022287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651cbc5-e7a4-40ee-b31c-1d024b655b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:40.023159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651cbc5-e7a4-40ee-b31c-1d024b655b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:40.023435 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651cbc5-e7a4-40ee-b31c-1d024b655b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:40.023707 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651cbc5-e7a4-40ee-b31c-1d024b655b10 admin admin] PATH_INFO: `/v3/users/830fba9768f6484badeb8caa14d6ce51` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:40.024917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651cbc5-e7a4-40ee-b31c-1d024b655b10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=830fba9768f6484badeb8caa14d6ce51)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:40.026833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651cbc5-e7a4-40ee-b31c-1d024b655b10 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:40.028516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:40.079016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f651cbc5-e7a4-40ee-b31c-1d024b655b10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '830fba9768f6484badeb8caa14d6ce51'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:40.091388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f651cbc5-e7a4-40ee-b31c-1d024b655b10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '830fba9768f6484badeb8caa14d6ce51'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:40.100064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f651cbc5-e7a4-40ee-b31c-1d024b655b10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '830fba9768f6484badeb8caa14d6ce51'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:40.108221 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f651cbc5-e7a4-40ee-b31c-1d024b655b10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '830fba9768f6484badeb8caa14d6ce51'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:40.108788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f651cbc5-e7a4-40ee-b31c-1d024b655b10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '830fba9768f6484badeb8caa14d6ce51'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:40.115789 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1161/2295] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:07:39 2022] DELETE /identity/v3/users/830fba9768f6484badeb8caa14d6ce51 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:40.120926 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc4d95-81ad-47d5-b952-a602b9a59bd0 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:40.121458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc4d95-81ad-47d5-b952-a602b9a59bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:40.121609 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc4d95-81ad-47d5-b952-a602b9a59bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:40.142846 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-a7bc4d95-81ad-47d5-b952-a602b9a59bd0 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:40.144083 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a7bc4d95-81ad-47d5-b952-a602b9a59bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:40.219190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1135/2296] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:40.230381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651cbc5-e7a4-40ee-b31c-1d024b655b10 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:40.232114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-f651cbc5-e7a4-40ee-b31c-1d024b655b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:40.311981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651cbc5-e7a4-40ee-b31c-1d024b655b10 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:40.316816 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40470e-a718-424b-b1f4-e392e9debae6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:40.317532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40470e-a718-424b-b1f4-e392e9debae6 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:40.317835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40470e-a718-424b-b1f4-e392e9debae6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:40.320101 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40470e-a718-424b-b1f4-e392e9debae6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:40.332844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40470e-a718-424b-b1f4-e392e9debae6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:40.335070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40470e-a718-424b-b1f4-e392e9debae6 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:40.347442 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1162/2297] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:07:40.940703 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc4d95-81ad-47d5-b952-a602b9a59bd0 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:40.943767 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a7bc4d95-81ad-47d5-b952-a602b9a59bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:40.976842 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc4d95-81ad-47d5-b952-a602b9a59bd0 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:40.983488 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:40.984373 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:40.984655 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:40.985159 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f admin admin] PATH_INFO: `/v3/projects/51584323f85e49ab98108553b7cd353c` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:40.986611 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=51584323f85e49ab98108553b7cd353c)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:40.988826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:41.081615 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51584323f85e49ab98108553b7cd353c', 'request_id': 'req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:41.088854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51584323f85e49ab98108553b7cd353c', 'request_id': 'req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:41.103461 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51584323f85e49ab98108553b7cd353c', 'request_id': 'req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:41.113833 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f admin admin] The token cache is being invalidate because project 51584323f85e49ab98108553b7cd353c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:07:41.114128 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:41.114809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:41.117164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1136/2298] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:07:40 2022] DELETE /identity/v3/projects/51584323f85e49ab98108553b7cd353c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:41.121918 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40470e-a718-424b-b1f4-e392e9debae6 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:41.133671 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40470e-a718-424b-b1f4-e392e9debae6 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:41.210998 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d862a-9000-4575-935c-9d1bfbe4ea98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:41.211872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d862a-9000-4575-935c-9d1bfbe4ea98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:41.212157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d862a-9000-4575-935c-9d1bfbe4ea98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:41.212436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d862a-9000-4575-935c-9d1bfbe4ea98 admin admin] PATH_INFO: `/v3/users/eff3cdaa69f34fcfb7f8198ebcb8faa4` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:41.213712 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d862a-9000-4575-935c-9d1bfbe4ea98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eff3cdaa69f34fcfb7f8198ebcb8faa4)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:41.215685 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d862a-9000-4575-935c-9d1bfbe4ea98 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:41.239244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:41.266513 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-4c3d862a-9000-4575-935c-9d1bfbe4ea98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eff3cdaa69f34fcfb7f8198ebcb8faa4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:41.278743 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-4c3d862a-9000-4575-935c-9d1bfbe4ea98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eff3cdaa69f34fcfb7f8198ebcb8faa4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:41.285632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-4c3d862a-9000-4575-935c-9d1bfbe4ea98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eff3cdaa69f34fcfb7f8198ebcb8faa4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:41.295202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-4c3d862a-9000-4575-935c-9d1bfbe4ea98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eff3cdaa69f34fcfb7f8198ebcb8faa4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:41.295812 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-4c3d862a-9000-4575-935c-9d1bfbe4ea98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eff3cdaa69f34fcfb7f8198ebcb8faa4'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:41.301793 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1163/2299] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:07:41 2022] DELETE /identity/v3/users/eff3cdaa69f34fcfb7f8198ebcb8faa4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:07:41.882118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:41.884307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:41.902530 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cea9eb-5426-40eb-b187-1f7a0165eb4f admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:41.959847 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:41.969592 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78a263-e96f-4be3-8efe-18cc114ed5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:41.970807 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78a263-e96f-4be3-8efe-18cc114ed5ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:41.971093 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78a263-e96f-4be3-8efe-18cc114ed5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:41.971518 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78a263-e96f-4be3-8efe-18cc114ed5ae admin admin] PATH_INFO: `/v3/projects/2566f0119f394602a92602df9532cadf` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:41.973029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78a263-e96f-4be3-8efe-18cc114ed5ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=2566f0119f394602a92602df9532cadf)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:41.975749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78a263-e96f-4be3-8efe-18cc114ed5ae admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:42.025548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6e78a263-e96f-4be3-8efe-18cc114ed5ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2566f0119f394602a92602df9532cadf', 'request_id': 'req-6e78a263-e96f-4be3-8efe-18cc114ed5ae'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:42.031803 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6e78a263-e96f-4be3-8efe-18cc114ed5ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2566f0119f394602a92602df9532cadf', 'request_id': 'req-6e78a263-e96f-4be3-8efe-18cc114ed5ae'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:42.040137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6e78a263-e96f-4be3-8efe-18cc114ed5ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2566f0119f394602a92602df9532cadf', 'request_id': 'req-6e78a263-e96f-4be3-8efe-18cc114ed5ae'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:42.055379 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6e78a263-e96f-4be3-8efe-18cc114ed5ae admin admin] The token cache is being invalidate because project 2566f0119f394602a92602df9532cadf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:07:42.057193 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6e78a263-e96f-4be3-8efe-18cc114ed5ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:42.057882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6e78a263-e96f-4be3-8efe-18cc114ed5ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:42.063358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1137/2300] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:07:41 2022] DELETE /identity/v3/projects/2566f0119f394602a92602df9532cadf => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:42.064519 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d862a-9000-4575-935c-9d1bfbe4ea98 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:42.066039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-4c3d862a-9000-4575-935c-9d1bfbe4ea98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:42.079677 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d862a-9000-4575-935c-9d1bfbe4ea98 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:42.147409 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f0747-9082-45ac-ab4a-52331e069aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:42.149375 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f0747-9082-45ac-ab4a-52331e069aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:42.149765 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f0747-9082-45ac-ab4a-52331e069aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:42.150147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f0747-9082-45ac-ab4a-52331e069aa3 admin admin] PATH_INFO: `/v3/users/eee2d98b64d44b14a58d3bac3a507283` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:42.152977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f0747-9082-45ac-ab4a-52331e069aa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eee2d98b64d44b14a58d3bac3a507283)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:42.156416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f0747-9082-45ac-ab4a-52331e069aa3 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:42.230518 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-608f0747-9082-45ac-ab4a-52331e069aa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eee2d98b64d44b14a58d3bac3a507283'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:42.246155 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-608f0747-9082-45ac-ab4a-52331e069aa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eee2d98b64d44b14a58d3bac3a507283'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:42.253648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-608f0747-9082-45ac-ab4a-52331e069aa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eee2d98b64d44b14a58d3bac3a507283'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:42.263122 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-608f0747-9082-45ac-ab4a-52331e069aa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eee2d98b64d44b14a58d3bac3a507283'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:42.263875 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-608f0747-9082-45ac-ab4a-52331e069aa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eee2d98b64d44b14a58d3bac3a507283'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:42.270475 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1164/2301] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:07:42 2022] DELETE /identity/v3/users/eee2d98b64d44b14a58d3bac3a507283 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:07:42.572342 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78a263-e96f-4be3-8efe-18cc114ed5ae admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:42.574744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6e78a263-e96f-4be3-8efe-18cc114ed5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:42.593409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78a263-e96f-4be3-8efe-18cc114ed5ae admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:42.658439 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c272c67-c8e0-43bd-aa4d-bf0389546b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:42.659229 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c272c67-c8e0-43bd-aa4d-bf0389546b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:42.659514 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c272c67-c8e0-43bd-aa4d-bf0389546b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:42.659796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c272c67-c8e0-43bd-aa4d-bf0389546b0e admin admin] PATH_INFO: `/v3/projects/e4f716ec22b24a128884fde16e969a07` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:42.661073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c272c67-c8e0-43bd-aa4d-bf0389546b0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4f716ec22b24a128884fde16e969a07)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:42.663310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c272c67-c8e0-43bd-aa4d-bf0389546b0e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:42.714004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2c272c67-c8e0-43bd-aa4d-bf0389546b0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4f716ec22b24a128884fde16e969a07', 'request_id': 'req-2c272c67-c8e0-43bd-aa4d-bf0389546b0e'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:42.720112 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2c272c67-c8e0-43bd-aa4d-bf0389546b0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4f716ec22b24a128884fde16e969a07', 'request_id': 'req-2c272c67-c8e0-43bd-aa4d-bf0389546b0e'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:42.734714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2c272c67-c8e0-43bd-aa4d-bf0389546b0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4f716ec22b24a128884fde16e969a07', 'request_id': 'req-2c272c67-c8e0-43bd-aa4d-bf0389546b0e'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:42.746318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2c272c67-c8e0-43bd-aa4d-bf0389546b0e admin admin] The token cache is being invalidate because project e4f716ec22b24a128884fde16e969a07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:07:42.746599 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2c272c67-c8e0-43bd-aa4d-bf0389546b0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:42.747245 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2c272c67-c8e0-43bd-aa4d-bf0389546b0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:07:42.748805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1138/2302] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:07:42 2022] DELETE /identity/v3/projects/e4f716ec22b24a128884fde16e969a07 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:07:42.753885 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bd3ea-5ada-4e07-a497-5f9fe9fca3a6 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:42.754167 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bd3ea-5ada-4e07-a497-5f9fe9fca3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:42.754439 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bd3ea-5ada-4e07-a497-5f9fe9fca3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:42.773436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-935bd3ea-5ada-4e07-a497-5f9fe9fca3a6 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:42.774781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-935bd3ea-5ada-4e07-a497-5f9fe9fca3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:42.792281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:42.849335 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1165/2303] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:42.857997 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434d385-e0ab-4120-894f-b88f56b2cd40 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:42.858308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434d385-e0ab-4120-894f-b88f56b2cd40 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:42.858640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434d385-e0ab-4120-894f-b88f56b2cd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:42.882642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-e434d385-e0ab-4120-894f-b88f56b2cd40 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:42.884054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e434d385-e0ab-4120-894f-b88f56b2cd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:42.893213 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1139/2304] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:42.898350 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bd3ea-5ada-4e07-a497-5f9fe9fca3a6 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:42.908437 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bd3ea-5ada-4e07-a497-5f9fe9fca3a6 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:42.913160 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1879b34-b851-4c6d-b803-72791ff638fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:42.913762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1879b34-b851-4c6d-b803-72791ff638fd admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:42.914055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1879b34-b851-4c6d-b803-72791ff638fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:42.914337 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1879b34-b851-4c6d-b803-72791ff638fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:42.915074 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1879b34-b851-4c6d-b803-72791ff638fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:07:42.915388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1879b34-b851-4c6d-b803-72791ff638fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:42.917092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1879b34-b851-4c6d-b803-72791ff638fd admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:42.931490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1166/2305] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:07:42.935592 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434d385-e0ab-4120-894f-b88f56b2cd40 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:42.945264 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434d385-e0ab-4120-894f-b88f56b2cd40 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:42.949977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d72b82-717e-4283-974c-f2b7d3c07174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:42.950634 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d72b82-717e-4283-974c-f2b7d3c07174 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:42.950922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d72b82-717e-4283-974c-f2b7d3c07174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:42.951194 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d72b82-717e-4283-974c-f2b7d3c07174 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:42.951904 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d72b82-717e-4283-974c-f2b7d3c07174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:42.953827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d72b82-717e-4283-974c-f2b7d3c07174 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:42.963120 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:42.985343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1140/2306] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:07:42 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:07:42.989465 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1879b34-b851-4c6d-b803-72791ff638fd admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:43.000534 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1879b34-b851-4c6d-b803-72791ff638fd admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:43.069965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e21022-a6de-4745-91ea-1d6ca306e40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:43.070674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e21022-a6de-4745-91ea-1d6ca306e40c admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:43.070975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e21022-a6de-4745-91ea-1d6ca306e40c admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:43.071239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e21022-a6de-4745-91ea-1d6ca306e40c admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:43.071952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e21022-a6de-4745-91ea-1d6ca306e40c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:43.073949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e21022-a6de-4745-91ea-1d6ca306e40c admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:43.103041 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1167/2307] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:07:42 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:07:43.104448 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d72b82-717e-4283-974c-f2b7d3c07174 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:43.116111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d72b82-717e-4283-974c-f2b7d3c07174 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:43.125086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd9733-76d9-41b1-b8e2-52eead17d48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:43.126067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd9733-76d9-41b1-b8e2-52eead17d48b admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:43.126350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd9733-76d9-41b1-b8e2-52eead17d48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:43.126632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd9733-76d9-41b1-b8e2-52eead17d48b admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:43.127380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd9733-76d9-41b1-b8e2-52eead17d48b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:43.129007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd9733-76d9-41b1-b8e2-52eead17d48b admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:43.147694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1141/2308] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:07:43.151676 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e21022-a6de-4745-91ea-1d6ca306e40c admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:43.165111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e21022-a6de-4745-91ea-1d6ca306e40c admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:43.170183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647482a-c50a-4a50-8b0b-c5c77fafaab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:43.170897 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647482a-c50a-4a50-8b0b-c5c77fafaab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:43.171227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647482a-c50a-4a50-8b0b-c5c77fafaab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:43.171513 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647482a-c50a-4a50-8b0b-c5c77fafaab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:43.172374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647482a-c50a-4a50-8b0b-c5c77fafaab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:43.174117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647482a-c50a-4a50-8b0b-c5c77fafaab4 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:43.188763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1168/2309] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:07:43.193254 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd9733-76d9-41b1-b8e2-52eead17d48b admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:43.206220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd9733-76d9-41b1-b8e2-52eead17d48b admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:43.212308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9b3e1-f1aa-4f5c-8ba2-1e8f68d3bf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:43.213285 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9b3e1-f1aa-4f5c-8ba2-1e8f68d3bf57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:43.213690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9b3e1-f1aa-4f5c-8ba2-1e8f68d3bf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:43.213961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9b3e1-f1aa-4f5c-8ba2-1e8f68d3bf57 admin admin] PATH_INFO: `/v3/projects/168a09155adc40b58961f1399f018a9c/users/a6909d098d884e6bb44f3a48d91199f2/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:43.232932 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e9b3e1-f1aa-4f5c-8ba2-1e8f68d3bf57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=168a09155adc40b58961f1399f018a9c, user_id=a6909d098d884e6bb44f3a48d91199f2, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:43.238351 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e9b3e1-f1aa-4f5c-8ba2-1e8f68d3bf57 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:43.256739 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1142/2310] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:07:43 2022] PUT /identity/v3/projects/168a09155adc40b58961f1399f018a9c/users/a6909d098d884e6bb44f3a48d91199f2/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:07:43.268105 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3db2a6-892f-4ec7-aee2-bc2c8932f538 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:43.268540 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3db2a6-892f-4ec7-aee2-bc2c8932f538 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:43.269029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3db2a6-892f-4ec7-aee2-bc2c8932f538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:43.310731 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-2f3db2a6-892f-4ec7-aee2-bc2c8932f538 None None] MFA Rules not processed for user `a6909d098d884e6bb44f3a48d91199f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:43.312015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2f3db2a6-892f-4ec7-aee2-bc2c8932f538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:43.394290 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1169/2311] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:43.408295 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9b3e1-f1aa-4f5c-8ba2-1e8f68d3bf57 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:43.410187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-19e9b3e1-f1aa-4f5c-8ba2-1e8f68d3bf57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:43.476947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9b3e1-f1aa-4f5c-8ba2-1e8f68d3bf57 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:43.480983 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff6929-5175-4d8e-9da8-59a95c5eeb04 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5c9ed6f9fd542069cf914003b636c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:43.481831 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6929-5175-4d8e-9da8-59a95c5eeb04 service glance] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:43.482352 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6929-5175-4d8e-9da8-59a95c5eeb04 service glance] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:43.482857 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6929-5175-4d8e-9da8-59a95c5eeb04 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:43.494030 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff6929-5175-4d8e-9da8-59a95c5eeb04 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:43.495510 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff6929-5175-4d8e-9da8-59a95c5eeb04 service glance] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:43.511108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1143/2312] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:07:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:07:47.039977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb595a-a9c1-4833-bc4c-643df68197b0 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:47.040511 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb595a-a9c1-4833-bc4c-643df68197b0 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:47.040806 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb595a-a9c1-4833-bc4c-643df68197b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:47.060033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-a0bb595a-a9c1-4833-bc4c-643df68197b0 None None] MFA Rules not processed for user `85dea53a4a034f5981b638bf92f2cf3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:47.061364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a0bb595a-a9c1-4833-bc4c-643df68197b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:47.073528 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:47.124564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1170/2313] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:07:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:07:47.795974 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77c62e-52a3-4aa8-9558-eb809d723817 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:47.795974 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77c62e-52a3-4aa8-9558-eb809d723817 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:47.795974 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77c62e-52a3-4aa8-9558-eb809d723817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:47.814289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-2d77c62e-52a3-4aa8-9558-eb809d723817 None None] MFA Rules not processed for user `8d6b823cc7dc434986f77739fa0259ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:07:47.815525 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-2d77c62e-52a3-4aa8-9558-eb809d723817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:47.885771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1144/2314] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Jul 12 22:07:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:07:47.948099 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb595a-a9c1-4833-bc4c-643df68197b0 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:07:47.950389 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a0bb595a-a9c1-4833-bc4c-643df68197b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:07:48.037517 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb595a-a9c1-4833-bc4c-643df68197b0 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:07:48.043704 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6952116-24da-48ac-9ad4-46f93960fe57 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837bb5ac18554dfdbbcc776689639494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:07:48.044667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6952116-24da-48ac-9ad4-46f93960fe57 service swift] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:07:48.044667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6952116-24da-48ac-9ad4-46f93960fe57 service swift] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:07:48.045034 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6952116-24da-48ac-9ad4-46f93960fe57 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:07:48.055546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6952116-24da-48ac-9ad4-46f93960fe57 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:07:48.057335 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6952116-24da-48ac-9ad4-46f93960fe57 service swift] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:07:48.066827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1171/2315] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Jul 12 22:07:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 22:07:57.881793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=70,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:07:58.062882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=50 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:08:01.543034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2096b3-1990-4b98-8b8f-a5bf5e88d40e None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:01.543034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2096b3-1990-4b98-8b8f-a5bf5e88d40e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:01.543647 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2096b3-1990-4b98-8b8f-a5bf5e88d40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:01.565305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-da2096b3-1990-4b98-8b8f-a5bf5e88d40e None None] MFA Rules not processed for user `8d6b823cc7dc434986f77739fa0259ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:08:01.567137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-da2096b3-1990-4b98-8b8f-a5bf5e88d40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:01.581640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1145/2316] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Jul 12 22:08:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:08:01.606761 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6952116-24da-48ac-9ad4-46f93960fe57 service swift] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:01.619450 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6952116-24da-48ac-9ad4-46f93960fe57 service swift] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:01.625076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63273ec9-75a3-4a6c-81f5-bff0c6d0c4ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837bb5ac18554dfdbbcc776689639494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:01.625901 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63273ec9-75a3-4a6c-81f5-bff0c6d0c4ee service swift] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:01.626193 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63273ec9-75a3-4a6c-81f5-bff0c6d0c4ee service swift] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:01.626515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63273ec9-75a3-4a6c-81f5-bff0c6d0c4ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:01.637085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63273ec9-75a3-4a6c-81f5-bff0c6d0c4ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:01.639014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63273ec9-75a3-4a6c-81f5-bff0c6d0c4ee service swift] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:01.649685 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1172/2317] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Jul 12 22:08:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 22:08:11.558771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:08:11.644771 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=6 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:08:13.025924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe6502-fc43-4d22-a9d8-77c906756c8d None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:13.026492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe6502-fc43-4d22-a9d8-77c906756c8d None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:13.026764 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe6502-fc43-4d22-a9d8-77c906756c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:13.055129 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-8dfe6502-fc43-4d22-a9d8-77c906756c8d None None] MFA Rules not processed for user `8d6b823cc7dc434986f77739fa0259ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:08:13.056567 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8dfe6502-fc43-4d22-a9d8-77c906756c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:13.064185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1146/2318] 10.222.0.44 () {58 vars in 967 bytes} [Tue Jul 12 22:08:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:08:13.082908 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63273ec9-75a3-4a6c-81f5-bff0c6d0c4ee service swift] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:13.097355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63273ec9-75a3-4a6c-81f5-bff0c6d0c4ee service swift] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:13.103367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d94302-b1ef-46e5-9b16-03263b699f5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837bb5ac18554dfdbbcc776689639494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:13.104050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d94302-b1ef-46e5-9b16-03263b699f5f service swift] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:13.104381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d94302-b1ef-46e5-9b16-03263b699f5f service swift] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:13.104664 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d94302-b1ef-46e5-9b16-03263b699f5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:13.115893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d94302-b1ef-46e5-9b16-03263b699f5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:13.117781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d94302-b1ef-46e5-9b16-03263b699f5f service swift] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:13.127728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1173/2319] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Jul 12 22:08:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 22:08:15.474866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe6502-fc43-4d22-a9d8-77c906756c8d None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:15.477544 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8dfe6502-fc43-4d22-a9d8-77c906756c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:15.567666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe6502-fc43-4d22-a9d8-77c906756c8d None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:15.572384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43585150-10a0-49f4-8497-427ad363bd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:15.573293 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43585150-10a0-49f4-8497-427ad363bd64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:15.573577 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43585150-10a0-49f4-8497-427ad363bd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:15.573853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43585150-10a0-49f4-8497-427ad363bd64 admin admin] PATH_INFO: `/v3/users/64f97c33dcc74dd4ad69c277415f4643` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:15.575069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43585150-10a0-49f4-8497-427ad363bd64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64f97c33dcc74dd4ad69c277415f4643)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:15.577041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43585150-10a0-49f4-8497-427ad363bd64 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:15.628657 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-43585150-10a0-49f4-8497-427ad363bd64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64f97c33dcc74dd4ad69c277415f4643'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:15.636129 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-43585150-10a0-49f4-8497-427ad363bd64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64f97c33dcc74dd4ad69c277415f4643'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:15.644691 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-43585150-10a0-49f4-8497-427ad363bd64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64f97c33dcc74dd4ad69c277415f4643'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:15.645395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-43585150-10a0-49f4-8497-427ad363bd64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64f97c33dcc74dd4ad69c277415f4643'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:15.646019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-43585150-10a0-49f4-8497-427ad363bd64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64f97c33dcc74dd4ad69c277415f4643'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:15.664803 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1147/2320] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:08:15 2022] DELETE /identity/v3/users/64f97c33dcc74dd4ad69c277415f4643 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:08:15.857119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d94302-b1ef-46e5-9b16-03263b699f5f service swift] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:15.869062 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d94302-b1ef-46e5-9b16-03263b699f5f service swift] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:15.928417 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8c06d-2fc3-435f-bbc0-07ac303f58d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:15.929325 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8c06d-2fc3-435f-bbc0-07ac303f58d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:15.929616 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8c06d-2fc3-435f-bbc0-07ac303f58d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:15.929900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8c06d-2fc3-435f-bbc0-07ac303f58d9 admin admin] PATH_INFO: `/v3/projects/304ae4bdbbf74ec9896a31d1f72ed72a` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:15.931086 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8c06d-2fc3-435f-bbc0-07ac303f58d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=304ae4bdbbf74ec9896a31d1f72ed72a)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:15.933022 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8c06d-2fc3-435f-bbc0-07ac303f58d9 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:15.980450 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-08d8c06d-2fc3-435f-bbc0-07ac303f58d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '304ae4bdbbf74ec9896a31d1f72ed72a', 'request_id': 'req-08d8c06d-2fc3-435f-bbc0-07ac303f58d9'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:15.985822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-08d8c06d-2fc3-435f-bbc0-07ac303f58d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '304ae4bdbbf74ec9896a31d1f72ed72a', 'request_id': 'req-08d8c06d-2fc3-435f-bbc0-07ac303f58d9'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:15.993349 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-08d8c06d-2fc3-435f-bbc0-07ac303f58d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '304ae4bdbbf74ec9896a31d1f72ed72a', 'request_id': 'req-08d8c06d-2fc3-435f-bbc0-07ac303f58d9'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:16.006940 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-08d8c06d-2fc3-435f-bbc0-07ac303f58d9 admin admin] The token cache is being invalidate because project 304ae4bdbbf74ec9896a31d1f72ed72a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:08:16.007054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-08d8c06d-2fc3-435f-bbc0-07ac303f58d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:16.007721 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-08d8c06d-2fc3-435f-bbc0-07ac303f58d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:16.009905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1174/2321] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:08:15 2022] DELETE /identity/v3/projects/304ae4bdbbf74ec9896a31d1f72ed72a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:08:16.015900 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43674aee-64e9-4498-9e02-59ad645eaac0 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:16.016327 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43674aee-64e9-4498-9e02-59ad645eaac0 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:16.016654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43674aee-64e9-4498-9e02-59ad645eaac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:16.038490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-43674aee-64e9-4498-9e02-59ad645eaac0 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:08:16.039501 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-43674aee-64e9-4498-9e02-59ad645eaac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:16.112516 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1148/2322] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:08:16.116393 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e6257-50f1-40e6-b865-a974ac8d8333 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:16.117524 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e6257-50f1-40e6-b865-a974ac8d8333 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:16.118088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e6257-50f1-40e6-b865-a974ac8d8333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:16.143847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-bb1e6257-50f1-40e6-b865-a974ac8d8333 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:08:16.145647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-bb1e6257-50f1-40e6-b865-a974ac8d8333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:16.154753 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1175/2323] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:08:16.158631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43674aee-64e9-4498-9e02-59ad645eaac0 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:16.169448 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43674aee-64e9-4498-9e02-59ad645eaac0 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:16.178204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545924fc-774f-4b26-84a9-064e8081bdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:16.179057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545924fc-774f-4b26-84a9-064e8081bdb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:16.179498 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545924fc-774f-4b26-84a9-064e8081bdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:16.179789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545924fc-774f-4b26-84a9-064e8081bdb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:16.180566 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545924fc-774f-4b26-84a9-064e8081bdb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:08:16.180933 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545924fc-774f-4b26-84a9-064e8081bdb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:16.182726 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545924fc-774f-4b26-84a9-064e8081bdb6 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:16.192263 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:08:16.200394 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1149/2324] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:08:16.201619 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e6257-50f1-40e6-b865-a974ac8d8333 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:16.212426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e6257-50f1-40e6-b865-a974ac8d8333 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:16.217421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ae807-b908-4bd5-b013-edcaa617f534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:16.218081 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ae807-b908-4bd5-b013-edcaa617f534 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:16.218373 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ae807-b908-4bd5-b013-edcaa617f534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:16.218652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ae807-b908-4bd5-b013-edcaa617f534 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:16.219363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ae807-b908-4bd5-b013-edcaa617f534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:16.221908 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ae807-b908-4bd5-b013-edcaa617f534 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:16.250901 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1176/2325] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:08:16 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:08:16.254628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545924fc-774f-4b26-84a9-064e8081bdb6 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:16.264840 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545924fc-774f-4b26-84a9-064e8081bdb6 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:16.331298 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebda2a9-d424-45c4-ae66-ee1c70f43c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:16.332403 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebda2a9-d424-45c4-ae66-ee1c70f43c49 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:16.333185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebda2a9-d424-45c4-ae66-ee1c70f43c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:16.333473 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebda2a9-d424-45c4-ae66-ee1c70f43c49 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:16.334421 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebda2a9-d424-45c4-ae66-ee1c70f43c49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:16.336396 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebda2a9-d424-45c4-ae66-ee1c70f43c49 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:16.378394 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1150/2326] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:08:16 2022] POST /identity/v3/users => generated 282 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:08:16.380390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ae807-b908-4bd5-b013-edcaa617f534 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:16.391074 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ae807-b908-4bd5-b013-edcaa617f534 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:16.396983 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c969f-cbc0-4f2b-b241-a9eea99f7367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:16.397666 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c969f-cbc0-4f2b-b241-a9eea99f7367 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:16.397953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c969f-cbc0-4f2b-b241-a9eea99f7367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:16.398234 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c969f-cbc0-4f2b-b241-a9eea99f7367 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:16.399117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c969f-cbc0-4f2b-b241-a9eea99f7367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:16.400864 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c969f-cbc0-4f2b-b241-a9eea99f7367 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:16.415516 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1177/2327] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:08:16.419760 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebda2a9-d424-45c4-ae66-ee1c70f43c49 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:16.430931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebda2a9-d424-45c4-ae66-ee1c70f43c49 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:16.442208 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3ce6c-eb80-4e93-bbd3-b6ba6eac3552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:16.443499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3ce6c-eb80-4e93-bbd3-b6ba6eac3552 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:16.445004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3ce6c-eb80-4e93-bbd3-b6ba6eac3552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:16.445523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3ce6c-eb80-4e93-bbd3-b6ba6eac3552 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:16.446371 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3ce6c-eb80-4e93-bbd3-b6ba6eac3552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:16.448093 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3ce6c-eb80-4e93-bbd3-b6ba6eac3552 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:16.459571 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1151/2328] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:08:16.464693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c969f-cbc0-4f2b-b241-a9eea99f7367 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:16.478031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c969f-cbc0-4f2b-b241-a9eea99f7367 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:16.483861 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17bd72-a40b-4a07-994e-13e53cb81d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:16.485048 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17bd72-a40b-4a07-994e-13e53cb81d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:16.485414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17bd72-a40b-4a07-994e-13e53cb81d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:16.485792 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17bd72-a40b-4a07-994e-13e53cb81d01 admin admin] PATH_INFO: `/v3/projects/c61f048fd7e34b5fb1176dcb7dd7e1cc/users/71c69de281b34c01a7be20650fd976bd/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:16.507559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17bd72-a40b-4a07-994e-13e53cb81d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61f048fd7e34b5fb1176dcb7dd7e1cc, user_id=71c69de281b34c01a7be20650fd976bd, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:16.510484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17bd72-a40b-4a07-994e-13e53cb81d01 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:16.534999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1178/2329] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:08:16 2022] PUT /identity/v3/projects/c61f048fd7e34b5fb1176dcb7dd7e1cc/users/71c69de281b34c01a7be20650fd976bd/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:08:16.545669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274aaf4-ec69-4a9b-a184-b4b468012dd9 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:16.546205 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274aaf4-ec69-4a9b-a184-b4b468012dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:16.546205 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274aaf4-ec69-4a9b-a184-b4b468012dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:16.581391 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-b274aaf4-ec69-4a9b-a184-b4b468012dd9 None None] MFA Rules not processed for user `71c69de281b34c01a7be20650fd976bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:08:16.582875 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b274aaf4-ec69-4a9b-a184-b4b468012dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:16.674541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1152/2330] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:08:16 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:08:16.685744 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17bd72-a40b-4a07-994e-13e53cb81d01 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:16.687437 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-db17bd72-a40b-4a07-994e-13e53cb81d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:16.702495 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:08:16.748816 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17bd72-a40b-4a07-994e-13e53cb81d01 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:16.753101 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91623aa1-6563-494e-958b-850c863e1e2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:16.753777 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91623aa1-6563-494e-958b-850c863e1e2b service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:16.754213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91623aa1-6563-494e-958b-850c863e1e2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:16.754965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91623aa1-6563-494e-958b-850c863e1e2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:16.763960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91623aa1-6563-494e-958b-850c863e1e2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:16.765441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91623aa1-6563-494e-958b-850c863e1e2b service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:16.779772 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1179/2331] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:08:16 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:08:20.535380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b997a-69b2-4a52-85dd-e867bad9b873 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:20.536762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b997a-69b2-4a52-85dd-e867bad9b873 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:20.536762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b997a-69b2-4a52-85dd-e867bad9b873 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:20.537866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1153/2332] 10.222.0.44 () {54 vars in 921 bytes} [Tue Jul 12 22:08:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:08:20.544604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71173f6c-792d-4960-8385-9131d8d9c204 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:20.545159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71173f6c-792d-4960-8385-9131d8d9c204 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:20.545463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71173f6c-792d-4960-8385-9131d8d9c204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:20.568698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-71173f6c-792d-4960-8385-9131d8d9c204 None None] MFA Rules not processed for user `24915625354f418eb7bc4d5ada85fdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:08:20.570311 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-71173f6c-792d-4960-8385-9131d8d9c204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:20.643978 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1180/2333] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Jul 12 22:08:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:08:20.655978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b997a-69b2-4a52-85dd-e867bad9b873 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:20.657806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-aa9b997a-69b2-4a52-85dd-e867bad9b873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:20.717026 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b997a-69b2-4a52-85dd-e867bad9b873 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:20.720585 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ee502-92a0-40cc-a2dc-fa29fe23a919 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5c9ed6f9fd542069cf914003b636c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:20.721253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ee502-92a0-40cc-a2dc-fa29fe23a919 service glance] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:20.721534 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ee502-92a0-40cc-a2dc-fa29fe23a919 service glance] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:20.721809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ee502-92a0-40cc-a2dc-fa29fe23a919 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:20.733039 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ee502-92a0-40cc-a2dc-fa29fe23a919 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:20.734535 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ee502-92a0-40cc-a2dc-fa29fe23a919 service glance] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:20.746925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1154/2334] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:08:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:08:20.818422 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03ba82-a2af-4d80-8a7b-e3292ef0e29a None None] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:20.818578 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03ba82-a2af-4d80-8a7b-e3292ef0e29a None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:20.818937 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03ba82-a2af-4d80-8a7b-e3292ef0e29a None None] PATH_INFO: `/` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:20.821900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1181/2335] 10.222.0.44 () {54 vars in 921 bytes} [Tue Jul 12 22:08:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:08:20.827879 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31b4b2-7c60-41e8-bcbf-5f4eac7f8bdf None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:20.828190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31b4b2-7c60-41e8-bcbf-5f4eac7f8bdf None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:20.828475 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31b4b2-7c60-41e8-bcbf-5f4eac7f8bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:20.847985 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-5f31b4b2-7c60-41e8-bcbf-5f4eac7f8bdf None None] MFA Rules not processed for user `53b891a734a64b5983564e207de0359d`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:08:20.849996 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-5f31b4b2-7c60-41e8-bcbf-5f4eac7f8bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:20.855619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:08:20.910227 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1155/2336] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Jul 12 22:08:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:08:20.924894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03ba82-a2af-4d80-8a7b-e3292ef0e29a None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:20.926686 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2e03ba82-a2af-4d80-8a7b-e3292ef0e29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:20.941791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03ba82-a2af-4d80-8a7b-e3292ef0e29a None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:20.945762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948df55-34a1-46bd-8b00-dde164050ab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:20.946414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948df55-34a1-46bd-8b00-dde164050ab1 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:20.946700 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948df55-34a1-46bd-8b00-dde164050ab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:20.946988 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948df55-34a1-46bd-8b00-dde164050ab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:20.954635 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948df55-34a1-46bd-8b00-dde164050ab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:20.956079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948df55-34a1-46bd-8b00-dde164050ab1 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:20.968161 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1182/2337] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:08:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:08:21.075377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb66fd0-2949-48ff-bf22-8a43ec42ff46 None None] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:21.075677 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb66fd0-2949-48ff-bf22-8a43ec42ff46 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:21.075960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb66fd0-2949-48ff-bf22-8a43ec42ff46 None None] PATH_INFO: `/` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:21.077513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1156/2338] 10.222.0.44 () {54 vars in 921 bytes} [Tue Jul 12 22:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 22:08:21.087614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e484c0-a857-4f95-ab15-67e029e80979 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:21.088337 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e484c0-a857-4f95-ab15-67e029e80979 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:21.088698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e484c0-a857-4f95-ab15-67e029e80979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:21.112752 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-e9e484c0-a857-4f95-ab15-67e029e80979 None None] MFA Rules not processed for user `24915625354f418eb7bc4d5ada85fdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:08:21.114628 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e9e484c0-a857-4f95-ab15-67e029e80979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:21.139629 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1183/2339] 10.222.0.44 () {58 vars in 1018 bytes} [Tue Jul 12 22:08:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:08:21.151878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb66fd0-2949-48ff-bf22-8a43ec42ff46 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:21.164605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb66fd0-2949-48ff-bf22-8a43ec42ff46 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:21.168667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c921f9-11fc-465f-b317-6be14cdd9571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:21.169489 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c921f9-11fc-465f-b317-6be14cdd9571 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:21.169808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c921f9-11fc-465f-b317-6be14cdd9571 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:21.172832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c921f9-11fc-465f-b317-6be14cdd9571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:21.182724 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c921f9-11fc-465f-b317-6be14cdd9571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:21.184751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c921f9-11fc-465f-b317-6be14cdd9571 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:21.199742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1157/2340] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:08:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:08:31.104941 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=60 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:08:31.192228 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=24 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:08:41.771621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1c2f0-1d46-4d37-a067-c3f86faae065 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:41.772309 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1c2f0-1d46-4d37-a067-c3f86faae065 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:41.772309 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1c2f0-1d46-4d37-a067-c3f86faae065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:41.790085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-d3e1c2f0-1d46-4d37-a067-c3f86faae065 None None] MFA Rules not processed for user `8d6b823cc7dc434986f77739fa0259ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:08:41.791352 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d3e1c2f0-1d46-4d37-a067-c3f86faae065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:41.847573 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1184/2341] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Jul 12 22:08:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:08:41.860106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c921f9-11fc-465f-b317-6be14cdd9571 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:41.861640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-07c921f9-11fc-465f-b317-6be14cdd9571 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:41.952883 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c921f9-11fc-465f-b317-6be14cdd9571 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:41.958180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ba240-3784-45f4-be53-df66ebb545a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837bb5ac18554dfdbbcc776689639494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:41.958884 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ba240-3784-45f4-be53-df66ebb545a7 service swift] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:41.959177 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ba240-3784-45f4-be53-df66ebb545a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:41.959458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ba240-3784-45f4-be53-df66ebb545a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:41.969271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18ba240-3784-45f4-be53-df66ebb545a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:41.970924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18ba240-3784-45f4-be53-df66ebb545a7 service swift] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:41.981744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1158/2342] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Jul 12 22:08:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 22:08:42.217156 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1c2f0-1d46-4d37-a067-c3f86faae065 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:42.219463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d3e1c2f0-1d46-4d37-a067-c3f86faae065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:42.339927 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1c2f0-1d46-4d37-a067-c3f86faae065 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:42.351126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c5fb7-1525-43e6-8768-d6548fe12392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:42.352117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c5fb7-1525-43e6-8768-d6548fe12392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:42.352913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c5fb7-1525-43e6-8768-d6548fe12392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:42.353200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c5fb7-1525-43e6-8768-d6548fe12392 admin admin] PATH_INFO: `/v3/users/a6909d098d884e6bb44f3a48d91199f2` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:42.354564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c5fb7-1525-43e6-8768-d6548fe12392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6909d098d884e6bb44f3a48d91199f2)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:42.363046 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c5fb7-1525-43e6-8768-d6548fe12392 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:42.443264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-178c5fb7-1525-43e6-8768-d6548fe12392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6909d098d884e6bb44f3a48d91199f2'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:42.463455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-178c5fb7-1525-43e6-8768-d6548fe12392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6909d098d884e6bb44f3a48d91199f2'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:42.475711 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-178c5fb7-1525-43e6-8768-d6548fe12392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6909d098d884e6bb44f3a48d91199f2'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:42.489555 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-178c5fb7-1525-43e6-8768-d6548fe12392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6909d098d884e6bb44f3a48d91199f2'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:42.490796 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-178c5fb7-1525-43e6-8768-d6548fe12392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6909d098d884e6bb44f3a48d91199f2'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:42.500093 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1185/2343] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:08:42 2022] DELETE /identity/v3/users/a6909d098d884e6bb44f3a48d91199f2 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:08:42.505015 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7227754-4ae0-4924-938d-90668ce5b3c7 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:42.505564 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7227754-4ae0-4924-938d-90668ce5b3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:42.506033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7227754-4ae0-4924-938d-90668ce5b3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:42.526271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-a7227754-4ae0-4924-938d-90668ce5b3c7 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:08:42.527515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a7227754-4ae0-4924-938d-90668ce5b3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:42.613395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1159/2344] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:08:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:08:42.621788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c5fb7-1525-43e6-8768-d6548fe12392 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:42.623337 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-178c5fb7-1525-43e6-8768-d6548fe12392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:42.653269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:08:42.697344 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c5fb7-1525-43e6-8768-d6548fe12392 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:42.701264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c08312-4a68-4303-9485-5aad1c7bd048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:42.702017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c08312-4a68-4303-9485-5aad1c7bd048 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:42.702365 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c08312-4a68-4303-9485-5aad1c7bd048 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:42.702724 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c08312-4a68-4303-9485-5aad1c7bd048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:42.713728 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c08312-4a68-4303-9485-5aad1c7bd048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:42.715262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c08312-4a68-4303-9485-5aad1c7bd048 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:42.726612 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1186/2345] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:08:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:08:43.053403 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7227754-4ae0-4924-938d-90668ce5b3c7 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:43.054882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a7227754-4ae0-4924-938d-90668ce5b3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:43.087418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7227754-4ae0-4924-938d-90668ce5b3c7 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:43.094041 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff01b2a-c95e-49e6-b512-0357bdb4a083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:43.094930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff01b2a-c95e-49e6-b512-0357bdb4a083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:43.095214 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff01b2a-c95e-49e6-b512-0357bdb4a083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:43.095492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff01b2a-c95e-49e6-b512-0357bdb4a083 admin admin] PATH_INFO: `/v3/projects/168a09155adc40b58961f1399f018a9c` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:43.096736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff01b2a-c95e-49e6-b512-0357bdb4a083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=168a09155adc40b58961f1399f018a9c)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:43.101011 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff01b2a-c95e-49e6-b512-0357bdb4a083 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:43.204011 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2ff01b2a-c95e-49e6-b512-0357bdb4a083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '168a09155adc40b58961f1399f018a9c', 'request_id': 'req-2ff01b2a-c95e-49e6-b512-0357bdb4a083'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:43.212011 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2ff01b2a-c95e-49e6-b512-0357bdb4a083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '168a09155adc40b58961f1399f018a9c', 'request_id': 'req-2ff01b2a-c95e-49e6-b512-0357bdb4a083'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:43.224859 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2ff01b2a-c95e-49e6-b512-0357bdb4a083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '168a09155adc40b58961f1399f018a9c', 'request_id': 'req-2ff01b2a-c95e-49e6-b512-0357bdb4a083'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:43.242844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2ff01b2a-c95e-49e6-b512-0357bdb4a083 admin admin] The token cache is being invalidate because project 168a09155adc40b58961f1399f018a9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:08:43.243335 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2ff01b2a-c95e-49e6-b512-0357bdb4a083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:43.244797 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-2ff01b2a-c95e-49e6-b512-0357bdb4a083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:08:43.246853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1160/2346] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:08:43 2022] DELETE /identity/v3/projects/168a09155adc40b58961f1399f018a9c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:08:43.251589 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e990c6-6fb0-4c03-be9b-27498841cc27 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:43.252106 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e990c6-6fb0-4c03-be9b-27498841cc27 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:43.252257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e990c6-6fb0-4c03-be9b-27498841cc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:43.272172 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-55e990c6-6fb0-4c03-be9b-27498841cc27 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:08:43.273436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-55e990c6-6fb0-4c03-be9b-27498841cc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:43.384846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1187/2347] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:08:43.392660 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116da3c-e690-4055-a1e8-1da26587249e None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:43.393059 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116da3c-e690-4055-a1e8-1da26587249e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:43.393245 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116da3c-e690-4055-a1e8-1da26587249e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:43.428702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-8116da3c-e690-4055-a1e8-1da26587249e None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:08:43.442482 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8116da3c-e690-4055-a1e8-1da26587249e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:43.453634 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1161/2348] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:08:43.458527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e990c6-6fb0-4c03-be9b-27498841cc27 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:43.476876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e990c6-6fb0-4c03-be9b-27498841cc27 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:43.483249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091a835-5d3a-4652-a437-e0b2f498bab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:43.484161 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091a835-5d3a-4652-a437-e0b2f498bab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:43.484663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091a835-5d3a-4652-a437-e0b2f498bab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:43.485213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091a835-5d3a-4652-a437-e0b2f498bab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:43.485999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091a835-5d3a-4652-a437-e0b2f498bab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:08:43.486318 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091a835-5d3a-4652-a437-e0b2f498bab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:43.488977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091a835-5d3a-4652-a437-e0b2f498bab8 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:43.508653 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1188/2349] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:08:43.509699 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116da3c-e690-4055-a1e8-1da26587249e None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:43.535760 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116da3c-e690-4055-a1e8-1da26587249e None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:43.543886 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5271ec-4a94-41c1-a3f6-5547b429b223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:43.550778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5271ec-4a94-41c1-a3f6-5547b429b223 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:43.550778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5271ec-4a94-41c1-a3f6-5547b429b223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:43.550778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5271ec-4a94-41c1-a3f6-5547b429b223 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:43.550778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5271ec-4a94-41c1-a3f6-5547b429b223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:43.550778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5271ec-4a94-41c1-a3f6-5547b429b223 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:43.554711 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:08:43.607869 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1162/2350] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:08:43 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:08:43.610566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091a835-5d3a-4652-a437-e0b2f498bab8 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:43.621607 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091a835-5d3a-4652-a437-e0b2f498bab8 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:43.700860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687c6dc-21d8-4f21-aa9e-855013d8d4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:43.701908 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687c6dc-21d8-4f21-aa9e-855013d8d4e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:43.702294 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687c6dc-21d8-4f21-aa9e-855013d8d4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:43.702678 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687c6dc-21d8-4f21-aa9e-855013d8d4e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:43.703487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687c6dc-21d8-4f21-aa9e-855013d8d4e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:43.705608 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687c6dc-21d8-4f21-aa9e-855013d8d4e1 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:43.727608 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1189/2351] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:08:43 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:08:43.731526 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5271ec-4a94-41c1-a3f6-5547b429b223 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:43.742967 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5271ec-4a94-41c1-a3f6-5547b429b223 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:43.748503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4589dec-70f1-4bab-90d3-263bdb0002a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:43.749236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4589dec-70f1-4bab-90d3-263bdb0002a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:43.749525 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4589dec-70f1-4bab-90d3-263bdb0002a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:43.749803 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4589dec-70f1-4bab-90d3-263bdb0002a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:43.750534 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4589dec-70f1-4bab-90d3-263bdb0002a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:43.752141 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4589dec-70f1-4bab-90d3-263bdb0002a0 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:43.763823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1163/2352] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:08:43.767662 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687c6dc-21d8-4f21-aa9e-855013d8d4e1 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:43.778834 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687c6dc-21d8-4f21-aa9e-855013d8d4e1 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:43.786329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56d543-0319-4992-b840-f50060df4f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:43.787303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56d543-0319-4992-b840-f50060df4f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:43.787619 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56d543-0319-4992-b840-f50060df4f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:43.787950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56d543-0319-4992-b840-f50060df4f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:43.788693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e56d543-0319-4992-b840-f50060df4f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:43.790657 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e56d543-0319-4992-b840-f50060df4f65 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:43.802779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1190/2353] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:08:43.806904 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4589dec-70f1-4bab-90d3-263bdb0002a0 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:43.817952 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4589dec-70f1-4bab-90d3-263bdb0002a0 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:43.823090 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8d716-8557-49df-ae65-1d5d5c5a255e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:43.824057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8d716-8557-49df-ae65-1d5d5c5a255e admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:43.824343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8d716-8557-49df-ae65-1d5d5c5a255e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:43.824620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8d716-8557-49df-ae65-1d5d5c5a255e admin admin] PATH_INFO: `/v3/projects/f86df0806e8747159530bd66cd36e7f3/users/dc1d7070c0db4eb4bba5e1e018da5bad/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:43.841595 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8d716-8557-49df-ae65-1d5d5c5a255e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f86df0806e8747159530bd66cd36e7f3, user_id=dc1d7070c0db4eb4bba5e1e018da5bad, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:43.844325 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8d716-8557-49df-ae65-1d5d5c5a255e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:43.859686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1164/2354] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:08:43 2022] PUT /identity/v3/projects/f86df0806e8747159530bd66cd36e7f3/users/dc1d7070c0db4eb4bba5e1e018da5bad/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:08:43.871415 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f4b33-f8be-4560-9315-81f0ef646eaa None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:43.871707 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f4b33-f8be-4560-9315-81f0ef646eaa None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:43.871988 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f4b33-f8be-4560-9315-81f0ef646eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:43.904756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-a50f4b33-f8be-4560-9315-81f0ef646eaa None None] MFA Rules not processed for user `dc1d7070c0db4eb4bba5e1e018da5bad`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:08:43.906144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a50f4b33-f8be-4560-9315-81f0ef646eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:43.930116 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:08:43.994042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1191/2355] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:08:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:08:44.005234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8d716-8557-49df-ae65-1d5d5c5a255e admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:44.007031 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-40b8d716-8557-49df-ae65-1d5d5c5a255e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:44.070211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8d716-8557-49df-ae65-1d5d5c5a255e admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:44.074074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee13e6-fea3-4408-a2a6-de7ad2f585e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:44.074823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee13e6-fea3-4408-a2a6-de7ad2f585e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:44.075123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee13e6-fea3-4408-a2a6-de7ad2f585e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:44.075553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee13e6-fea3-4408-a2a6-de7ad2f585e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:44.086356 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee13e6-fea3-4408-a2a6-de7ad2f585e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:44.087930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee13e6-fea3-4408-a2a6-de7ad2f585e4 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:44.098569 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1165/2356] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:08:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:08:47.666278 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef941acf-4ff3-4028-8de7-49cbb5e15238 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:47.667048 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef941acf-4ff3-4028-8de7-49cbb5e15238 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:47.667048 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef941acf-4ff3-4028-8de7-49cbb5e15238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:47.695667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-ef941acf-4ff3-4028-8de7-49cbb5e15238 None None] MFA Rules not processed for user `dc1d7070c0db4eb4bba5e1e018da5bad`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:08:47.696937 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ef941acf-4ff3-4028-8de7-49cbb5e15238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:47.705104 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1192/2357] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:08:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:08:52.837366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee13e6-fea3-4408-a2a6-de7ad2f585e4 service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:08:52.838553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0dee13e6-fea3-4408-a2a6-de7ad2f585e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:52.913774 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee13e6-fea3-4408-a2a6-de7ad2f585e4 service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:08:52.918907 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07385923-4f95-4d91-8614-e0eec58d80d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f1ef52f8eb42a59a50a69873b9ffaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:08:52.919565 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07385923-4f95-4d91-8614-e0eec58d80d2 service placement] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:08:52.919856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07385923-4f95-4d91-8614-e0eec58d80d2 service placement] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:08:52.920130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07385923-4f95-4d91-8614-e0eec58d80d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:08:52.922177 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-07385923-4f95-4d91-8614-e0eec58d80d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:08:52.934869 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07385923-4f95-4d91-8614-e0eec58d80d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:08:52.936375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07385923-4f95-4d91-8614-e0eec58d80d2 service placement] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:08:52.946375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1166/2358] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:08:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:08:57.681629 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=20 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:09:02.941967 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=79,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:09:08.163415 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f9359-d8ff-4bf6-8fb2-2af7966fb949 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:08.163868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f9359-d8ff-4bf6-8fb2-2af7966fb949 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:08.163868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f9359-d8ff-4bf6-8fb2-2af7966fb949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:08.183941 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-ec9f9359-d8ff-4bf6-8fb2-2af7966fb949 None None] MFA Rules not processed for user `8d6b823cc7dc434986f77739fa0259ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:09:08.185206 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ec9f9359-d8ff-4bf6-8fb2-2af7966fb949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:08.244107 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1193/2359] 10.222.0.44 () {58 vars in 967 bytes} [Tue Jul 12 22:09:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:09:08.270404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07385923-4f95-4d91-8614-e0eec58d80d2 service placement] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:08.272603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-07385923-4f95-4d91-8614-e0eec58d80d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:08.375200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07385923-4f95-4d91-8614-e0eec58d80d2 service placement] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:08.381010 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b42543-1db4-4c4c-b30f-1db8e48fb5b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837bb5ac18554dfdbbcc776689639494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:08.381649 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b42543-1db4-4c4c-b30f-1db8e48fb5b6 service swift] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:08.381929 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b42543-1db4-4c4c-b30f-1db8e48fb5b6 service swift] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:08.382203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b42543-1db4-4c4c-b30f-1db8e48fb5b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:08.390966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b42543-1db4-4c4c-b30f-1db8e48fb5b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:08.392548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b42543-1db4-4c4c-b30f-1db8e48fb5b6 service swift] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:08.401367 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1167/2360] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Jul 12 22:09:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 22:09:10.650938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f9359-d8ff-4bf6-8fb2-2af7966fb949 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:10.652386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ec9f9359-d8ff-4bf6-8fb2-2af7966fb949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:10.724403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f9359-d8ff-4bf6-8fb2-2af7966fb949 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:10.729178 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ac5ff-d4c9-4ddf-93bc-bdd77eedfe9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:10.730061 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ac5ff-d4c9-4ddf-93bc-bdd77eedfe9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:10.730345 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ac5ff-d4c9-4ddf-93bc-bdd77eedfe9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:10.730621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ac5ff-d4c9-4ddf-93bc-bdd77eedfe9b admin admin] PATH_INFO: `/v3/users/71c69de281b34c01a7be20650fd976bd` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:10.732028 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ac5ff-d4c9-4ddf-93bc-bdd77eedfe9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=71c69de281b34c01a7be20650fd976bd)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:10.733985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ac5ff-d4c9-4ddf-93bc-bdd77eedfe9b admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:10.799472 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-7f8ac5ff-d4c9-4ddf-93bc-bdd77eedfe9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71c69de281b34c01a7be20650fd976bd'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:10.815615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-7f8ac5ff-d4c9-4ddf-93bc-bdd77eedfe9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71c69de281b34c01a7be20650fd976bd'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:10.824426 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-7f8ac5ff-d4c9-4ddf-93bc-bdd77eedfe9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71c69de281b34c01a7be20650fd976bd'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:10.835962 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-7f8ac5ff-d4c9-4ddf-93bc-bdd77eedfe9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71c69de281b34c01a7be20650fd976bd'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:10.836687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-7f8ac5ff-d4c9-4ddf-93bc-bdd77eedfe9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71c69de281b34c01a7be20650fd976bd'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:10.844639 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1194/2361] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:09:10 2022] DELETE /identity/v3/users/71c69de281b34c01a7be20650fd976bd => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:09:10.849315 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf415d5-58b7-4356-b5a4-def6a6c52a2a None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:10.849828 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf415d5-58b7-4356-b5a4-def6a6c52a2a None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:10.850037 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf415d5-58b7-4356-b5a4-def6a6c52a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:10.868956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-ddf415d5-58b7-4356-b5a4-def6a6c52a2a None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:09:10.870187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ddf415d5-58b7-4356-b5a4-def6a6c52a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:10.951755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1168/2362] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:09:10.962407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ac5ff-d4c9-4ddf-93bc-bdd77eedfe9b admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:10.964321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-7f8ac5ff-d4c9-4ddf-93bc-bdd77eedfe9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:11.011972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:09:11.058653 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ac5ff-d4c9-4ddf-93bc-bdd77eedfe9b admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:11.063794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ece619-6aa4-483e-bdcb-fa45c3015413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:11.064438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ece619-6aa4-483e-bdcb-fa45c3015413 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:11.064749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ece619-6aa4-483e-bdcb-fa45c3015413 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:11.065790 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ece619-6aa4-483e-bdcb-fa45c3015413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:11.079795 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ece619-6aa4-483e-bdcb-fa45c3015413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:11.081466 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ece619-6aa4-483e-bdcb-fa45c3015413 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:11.093004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1195/2363] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:09:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:09:11.688784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf415d5-58b7-4356-b5a4-def6a6c52a2a None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:11.691206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ddf415d5-58b7-4356-b5a4-def6a6c52a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:11.703373 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf415d5-58b7-4356-b5a4-def6a6c52a2a None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:11.708554 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65de8bf-ea3a-4c56-9e29-9fc3ed15f98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:11.709453 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65de8bf-ea3a-4c56-9e29-9fc3ed15f98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:11.709740 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65de8bf-ea3a-4c56-9e29-9fc3ed15f98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:11.710027 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65de8bf-ea3a-4c56-9e29-9fc3ed15f98b admin admin] PATH_INFO: `/v3/projects/c61f048fd7e34b5fb1176dcb7dd7e1cc` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:11.711244 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65de8bf-ea3a-4c56-9e29-9fc3ed15f98b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c61f048fd7e34b5fb1176dcb7dd7e1cc)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:11.713179 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65de8bf-ea3a-4c56-9e29-9fc3ed15f98b admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:11.784640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c65de8bf-ea3a-4c56-9e29-9fc3ed15f98b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c61f048fd7e34b5fb1176dcb7dd7e1cc', 'request_id': 'req-c65de8bf-ea3a-4c56-9e29-9fc3ed15f98b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:11.797437 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c65de8bf-ea3a-4c56-9e29-9fc3ed15f98b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c61f048fd7e34b5fb1176dcb7dd7e1cc', 'request_id': 'req-c65de8bf-ea3a-4c56-9e29-9fc3ed15f98b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:11.802947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c65de8bf-ea3a-4c56-9e29-9fc3ed15f98b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c61f048fd7e34b5fb1176dcb7dd7e1cc', 'request_id': 'req-c65de8bf-ea3a-4c56-9e29-9fc3ed15f98b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:11.817769 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c65de8bf-ea3a-4c56-9e29-9fc3ed15f98b admin admin] The token cache is being invalidate because project c61f048fd7e34b5fb1176dcb7dd7e1cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:09:11.818092 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c65de8bf-ea3a-4c56-9e29-9fc3ed15f98b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:11.818752 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c65de8bf-ea3a-4c56-9e29-9fc3ed15f98b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:11.820753 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1169/2364] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:09:11 2022] DELETE /identity/v3/projects/c61f048fd7e34b5fb1176dcb7dd7e1cc => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:09:11.825449 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862306cb-1010-4969-b0d5-4b49b70d5a6b None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:11.825962 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862306cb-1010-4969-b0d5-4b49b70d5a6b None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:11.826092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862306cb-1010-4969-b0d5-4b49b70d5a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:11.849246 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-862306cb-1010-4969-b0d5-4b49b70d5a6b None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:09:11.850936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-862306cb-1010-4969-b0d5-4b49b70d5a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:11.918582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1196/2365] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:09:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:09:11.926601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a04236-1a89-4b3a-901c-592f3fba0c41 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:11.927127 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a04236-1a89-4b3a-901c-592f3fba0c41 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:11.927231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a04236-1a89-4b3a-901c-592f3fba0c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:11.946483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-25a04236-1a89-4b3a-901c-592f3fba0c41 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:09:11.947956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-25a04236-1a89-4b3a-901c-592f3fba0c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:11.956148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1170/2366] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:09:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:09:11.960569 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862306cb-1010-4969-b0d5-4b49b70d5a6b None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:11.973040 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862306cb-1010-4969-b0d5-4b49b70d5a6b None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:11.977968 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5149da5-e0e6-4d0f-91ca-34dac1fe814b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:11.978636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5149da5-e0e6-4d0f-91ca-34dac1fe814b admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:11.978925 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5149da5-e0e6-4d0f-91ca-34dac1fe814b admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:11.979209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5149da5-e0e6-4d0f-91ca-34dac1fe814b admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:11.979950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5149da5-e0e6-4d0f-91ca-34dac1fe814b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:09:11.980400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5149da5-e0e6-4d0f-91ca-34dac1fe814b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:11.982273 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5149da5-e0e6-4d0f-91ca-34dac1fe814b admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:11.996207 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1197/2367] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:09:12.000895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a04236-1a89-4b3a-901c-592f3fba0c41 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:12.012347 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a04236-1a89-4b3a-901c-592f3fba0c41 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:12.018369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6dc19f-be3b-491f-b41c-f0d920af0417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:12.019138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6dc19f-be3b-491f-b41c-f0d920af0417 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:12.019427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6dc19f-be3b-491f-b41c-f0d920af0417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:12.019708 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6dc19f-be3b-491f-b41c-f0d920af0417 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:12.020451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6dc19f-be3b-491f-b41c-f0d920af0417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:12.024131 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6dc19f-be3b-491f-b41c-f0d920af0417 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:12.028197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:09:12.060037 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1171/2368] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:09:11 2022] POST /identity/v3/projects => generated 399 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:09:12.064198 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5149da5-e0e6-4d0f-91ca-34dac1fe814b admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:12.075532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5149da5-e0e6-4d0f-91ca-34dac1fe814b admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:12.136322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3995da-c17a-4908-9db4-1725670e97d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:12.137216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3995da-c17a-4908-9db4-1725670e97d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:12.137564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3995da-c17a-4908-9db4-1725670e97d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:12.137906 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3995da-c17a-4908-9db4-1725670e97d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:12.138879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3995da-c17a-4908-9db4-1725670e97d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:12.141168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3995da-c17a-4908-9db4-1725670e97d5 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:12.166024 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1198/2369] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:09:12 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:09:12.170157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6dc19f-be3b-491f-b41c-f0d920af0417 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:12.184052 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6dc19f-be3b-491f-b41c-f0d920af0417 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:12.190715 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7da82-3cdc-4944-a4bd-791c03d11faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:12.191416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7da82-3cdc-4944-a4bd-791c03d11faf admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:12.191701 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7da82-3cdc-4944-a4bd-791c03d11faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:12.191976 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7da82-3cdc-4944-a4bd-791c03d11faf admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:12.192727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7da82-3cdc-4944-a4bd-791c03d11faf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:12.194694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7da82-3cdc-4944-a4bd-791c03d11faf admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:12.208744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1172/2370] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:09:12.211133 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3995da-c17a-4908-9db4-1725670e97d5 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:12.223182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3995da-c17a-4908-9db4-1725670e97d5 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:12.228271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f10bb2-7969-4582-a781-a1b5cca59c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:12.228996 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f10bb2-7969-4582-a781-a1b5cca59c2e admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:12.229284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f10bb2-7969-4582-a781-a1b5cca59c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:12.229559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f10bb2-7969-4582-a781-a1b5cca59c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:12.230421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f10bb2-7969-4582-a781-a1b5cca59c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:12.232072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f10bb2-7969-4582-a781-a1b5cca59c2e admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:12.245929 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1199/2371] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:09:12.250404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7da82-3cdc-4944-a4bd-791c03d11faf admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:12.267345 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941ff2c-489e-48af-ab1c-c05d58abdacf None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:12.267630 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941ff2c-489e-48af-ab1c-c05d58abdacf None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:12.268001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941ff2c-489e-48af-ab1c-c05d58abdacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:12.269636 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7da82-3cdc-4944-a4bd-791c03d11faf admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:12.274484 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067d05e-991e-463a-9a24-6ad8232470ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:12.275508 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067d05e-991e-463a-9a24-6ad8232470ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:12.275801 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067d05e-991e-463a-9a24-6ad8232470ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:12.276123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067d05e-991e-463a-9a24-6ad8232470ec admin admin] PATH_INFO: `/v3/projects/9087f1a50b4e4cfd877d5701329126e6/users/02da7d0e526b49b19aeafb74cd7f667f/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:12.287777 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-d941ff2c-489e-48af-ab1c-c05d58abdacf None None] MFA Rules not processed for user `309de655c8d64223a9c2d75ff5618411`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:09:12.289145 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d941ff2c-489e-48af-ab1c-c05d58abdacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:12.294174 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067d05e-991e-463a-9a24-6ad8232470ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=9087f1a50b4e4cfd877d5701329126e6, user_id=02da7d0e526b49b19aeafb74cd7f667f, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:12.297560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067d05e-991e-463a-9a24-6ad8232470ec admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:12.311344 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1173/2372] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:09:12 2022] PUT /identity/v3/projects/9087f1a50b4e4cfd877d5701329126e6/users/02da7d0e526b49b19aeafb74cd7f667f/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:09:12.321844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:09:12.322668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b5135-50e1-4835-9241-264920a1db7f None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:12.323187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b5135-50e1-4835-9241-264920a1db7f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:12.324182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b5135-50e1-4835-9241-264920a1db7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:12.359445 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1200/2373] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:09:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:09:12.360068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-541b5135-50e1-4835-9241-264920a1db7f None None] MFA Rules not processed for user `02da7d0e526b49b19aeafb74cd7f667f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:09:12.362289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-541b5135-50e1-4835-9241-264920a1db7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:12.460886 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1174/2374] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:09:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:09:12.473686 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941ff2c-489e-48af-ab1c-c05d58abdacf None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:12.475333 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d941ff2c-489e-48af-ab1c-c05d58abdacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:12.539278 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941ff2c-489e-48af-ab1c-c05d58abdacf None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:12.592363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9285d31-432b-49f7-956a-c7e8e0949a08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:12.593049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9285d31-432b-49f7-956a-c7e8e0949a08 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:12.593485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9285d31-432b-49f7-956a-c7e8e0949a08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:12.593769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9285d31-432b-49f7-956a-c7e8e0949a08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:12.602749 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9285d31-432b-49f7-956a-c7e8e0949a08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:12.604179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9285d31-432b-49f7-956a-c7e8e0949a08 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:12.613965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1201/2375] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:09:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:09:20.459790 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c82d24-6ee2-4c8a-a2a1-2938b523703c None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:20.461185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c82d24-6ee2-4c8a-a2a1-2938b523703c None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:20.461731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c82d24-6ee2-4c8a-a2a1-2938b523703c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:20.483299 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-06c82d24-6ee2-4c8a-a2a1-2938b523703c None None] MFA Rules not processed for user `4c137f9412de4914b77732e1ac29ec36`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:09:20.484522 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-06c82d24-6ee2-4c8a-a2a1-2938b523703c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:20.547330 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1175/2376] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:09:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:09:21.831567 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9285d31-432b-49f7-956a-c7e8e0949a08 service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:21.960900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9285d31-432b-49f7-956a-c7e8e0949a08 service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:21.972543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4bc30-e3fe-4324-b374-d8ac552c4be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:21.973677 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4bc30-e3fe-4324-b374-d8ac552c4be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:21.974157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4bc30-e3fe-4324-b374-d8ac552c4be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:21.974544 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4bc30-e3fe-4324-b374-d8ac552c4be2 admin admin] PATH_INFO: `/v3/users/02da7d0e526b49b19aeafb74cd7f667f` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:21.976078 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4bc30-e3fe-4324-b374-d8ac552c4be2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02da7d0e526b49b19aeafb74cd7f667f)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:21.980591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4bc30-e3fe-4324-b374-d8ac552c4be2 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:22.059933 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:09:22.070780 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dac4bc30-e3fe-4324-b374-d8ac552c4be2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02da7d0e526b49b19aeafb74cd7f667f'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:22.090446 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dac4bc30-e3fe-4324-b374-d8ac552c4be2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02da7d0e526b49b19aeafb74cd7f667f'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:22.099632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dac4bc30-e3fe-4324-b374-d8ac552c4be2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02da7d0e526b49b19aeafb74cd7f667f'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:22.114924 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dac4bc30-e3fe-4324-b374-d8ac552c4be2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02da7d0e526b49b19aeafb74cd7f667f'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:22.115717 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dac4bc30-e3fe-4324-b374-d8ac552c4be2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02da7d0e526b49b19aeafb74cd7f667f'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:22.124748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1202/2377] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:09:21 2022] DELETE /identity/v3/users/02da7d0e526b49b19aeafb74cd7f667f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:09:22.129733 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cdf5d-ff0e-47fa-b769-63178509baa3 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:22.131042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cdf5d-ff0e-47fa-b769-63178509baa3 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:22.131500 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cdf5d-ff0e-47fa-b769-63178509baa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:22.177294 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-bd0cdf5d-ff0e-47fa-b769-63178509baa3 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:09:22.178525 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-bd0cdf5d-ff0e-47fa-b769-63178509baa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:22.253523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:09:22.287587 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1176/2378] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:09:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:09:22.302158 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4bc30-e3fe-4324-b374-d8ac552c4be2 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:22.305027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-dac4bc30-e3fe-4324-b374-d8ac552c4be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:22.427970 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4bc30-e3fe-4324-b374-d8ac552c4be2 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:22.433723 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7323e-7a87-4bb8-af0e-9147bf1d6a95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:22.435145 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7323e-7a87-4bb8-af0e-9147bf1d6a95 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:22.435533 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7323e-7a87-4bb8-af0e-9147bf1d6a95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:22.435912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7323e-7a87-4bb8-af0e-9147bf1d6a95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:22.449471 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae7323e-7a87-4bb8-af0e-9147bf1d6a95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:22.451054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae7323e-7a87-4bb8-af0e-9147bf1d6a95 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:22.519837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1203/2379] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:09:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:09:23.058814 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0cdf5d-ff0e-47fa-b769-63178509baa3 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:23.060013 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-bd0cdf5d-ff0e-47fa-b769-63178509baa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:23.081111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0cdf5d-ff0e-47fa-b769-63178509baa3 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:23.092477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b451f-f25d-4424-b0fd-1361c8774c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:23.093788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b451f-f25d-4424-b0fd-1361c8774c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:23.094236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b451f-f25d-4424-b0fd-1361c8774c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:23.094736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b451f-f25d-4424-b0fd-1361c8774c0e admin admin] PATH_INFO: `/v3/projects/9087f1a50b4e4cfd877d5701329126e6` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:23.097253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b451f-f25d-4424-b0fd-1361c8774c0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9087f1a50b4e4cfd877d5701329126e6)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:23.099735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b451f-f25d-4424-b0fd-1361c8774c0e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:23.187285 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-349b451f-f25d-4424-b0fd-1361c8774c0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9087f1a50b4e4cfd877d5701329126e6', 'request_id': 'req-349b451f-f25d-4424-b0fd-1361c8774c0e'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:23.203515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-349b451f-f25d-4424-b0fd-1361c8774c0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9087f1a50b4e4cfd877d5701329126e6', 'request_id': 'req-349b451f-f25d-4424-b0fd-1361c8774c0e'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:23.224315 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-349b451f-f25d-4424-b0fd-1361c8774c0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9087f1a50b4e4cfd877d5701329126e6', 'request_id': 'req-349b451f-f25d-4424-b0fd-1361c8774c0e'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:23.257517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-349b451f-f25d-4424-b0fd-1361c8774c0e admin admin] The token cache is being invalidate because project 9087f1a50b4e4cfd877d5701329126e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:09:23.259590 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-349b451f-f25d-4424-b0fd-1361c8774c0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:23.261029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-349b451f-f25d-4424-b0fd-1361c8774c0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:23.276809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1177/2380] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:09:23 2022] DELETE /identity/v3/projects/9087f1a50b4e4cfd877d5701329126e6 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:09:30.657219 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434f419-7026-498c-9ea9-1bfd7792d4a5 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:30.658029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434f419-7026-498c-9ea9-1bfd7792d4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:30.658371 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434f419-7026-498c-9ea9-1bfd7792d4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:30.676273 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-0434f419-7026-498c-9ea9-1bfd7792d4a5 None None] MFA Rules not processed for user `8d6b823cc7dc434986f77739fa0259ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:09:30.677534 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0434f419-7026-498c-9ea9-1bfd7792d4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:30.741694 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1204/2381] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Jul 12 22:09:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:09:30.756591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b451f-f25d-4424-b0fd-1361c8774c0e admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:30.758759 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-349b451f-f25d-4424-b0fd-1361c8774c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:30.886115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b451f-f25d-4424-b0fd-1361c8774c0e admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:30.892095 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16708c3-5808-4bc9-9e47-29d3130ccee0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837bb5ac18554dfdbbcc776689639494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:30.893108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16708c3-5808-4bc9-9e47-29d3130ccee0 service swift] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:30.893398 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16708c3-5808-4bc9-9e47-29d3130ccee0 service swift] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:30.893865 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16708c3-5808-4bc9-9e47-29d3130ccee0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:30.905296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16708c3-5808-4bc9-9e47-29d3130ccee0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:30.907206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16708c3-5808-4bc9-9e47-29d3130ccee0 service swift] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:30.926630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1178/2382] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Jul 12 22:09:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 22:09:31.567081 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e29a1-795a-4e0c-b8c2-80789f75f9d6 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:31.567453 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e29a1-795a-4e0c-b8c2-80789f75f9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:31.567657 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e29a1-795a-4e0c-b8c2-80789f75f9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:31.587972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-553e29a1-795a-4e0c-b8c2-80789f75f9d6 None None] MFA Rules not processed for user `8d6b823cc7dc434986f77739fa0259ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:09:31.589546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-553e29a1-795a-4e0c-b8c2-80789f75f9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:31.603880 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1205/2383] 10.222.0.44 () {58 vars in 967 bytes} [Tue Jul 12 22:09:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:09:31.617717 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16708c3-5808-4bc9-9e47-29d3130ccee0 service swift] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:31.629605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16708c3-5808-4bc9-9e47-29d3130ccee0 service swift] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:31.635899 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a29a6b-ed26-4459-b05d-69f92b8ee68c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837bb5ac18554dfdbbcc776689639494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:31.636654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a29a6b-ed26-4459-b05d-69f92b8ee68c service swift] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:31.636858 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a29a6b-ed26-4459-b05d-69f92b8ee68c service swift] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:31.637179 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a29a6b-ed26-4459-b05d-69f92b8ee68c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:31.647956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a29a6b-ed26-4459-b05d-69f92b8ee68c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:31.649609 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a29a6b-ed26-4459-b05d-69f92b8ee68c service swift] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:31.661292 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1179/2384] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Jul 12 22:09:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 22:09:31.901881 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b995f2-0bfa-44d1-9961-bf94e87c10cc None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:31.902464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b995f2-0bfa-44d1-9961-bf94e87c10cc None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:31.902464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b995f2-0bfa-44d1-9961-bf94e87c10cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:31.926644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-f1b995f2-0bfa-44d1-9961-bf94e87c10cc None None] MFA Rules not processed for user `8d6b823cc7dc434986f77739fa0259ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:09:31.930389 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-f1b995f2-0bfa-44d1-9961-bf94e87c10cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:31.951628 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1206/2385] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Jul 12 22:09:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:09:31.953284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a29a6b-ed26-4459-b05d-69f92b8ee68c service swift] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:31.967799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a29a6b-ed26-4459-b05d-69f92b8ee68c service swift] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:31.992342 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45edff-3043-4851-b360-5c86529b5769 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837bb5ac18554dfdbbcc776689639494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:31.993100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45edff-3043-4851-b360-5c86529b5769 service swift] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:31.993388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45edff-3043-4851-b360-5c86529b5769 service swift] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:31.993708 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45edff-3043-4851-b360-5c86529b5769 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:32.014039 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45edff-3043-4851-b360-5c86529b5769 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:32.016231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45edff-3043-4851-b360-5c86529b5769 service swift] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:32.027743 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1180/2386] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Jul 12 22:09:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 22:09:32.423454 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa88344-a24b-47a8-b184-6f5512b9aa8e None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:32.424440 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa88344-a24b-47a8-b184-6f5512b9aa8e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:32.424860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa88344-a24b-47a8-b184-6f5512b9aa8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:32.444504 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-1fa88344-a24b-47a8-b184-6f5512b9aa8e None None] MFA Rules not processed for user `8d6b823cc7dc434986f77739fa0259ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:09:32.445783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-1fa88344-a24b-47a8-b184-6f5512b9aa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:32.453204 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1207/2387] 10.222.0.44 () {58 vars in 967 bytes} [Tue Jul 12 22:09:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:09:32.466123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45edff-3043-4851-b360-5c86529b5769 service swift] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:32.477792 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45edff-3043-4851-b360-5c86529b5769 service swift] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:32.482969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e067a0-3350-480d-ab2c-1729846b3d25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837bb5ac18554dfdbbcc776689639494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:32.483688 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e067a0-3350-480d-ab2c-1729846b3d25 service swift] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:32.484175 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e067a0-3350-480d-ab2c-1729846b3d25 service swift] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:32.484531 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e067a0-3350-480d-ab2c-1729846b3d25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:32.495203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e067a0-3350-480d-ab2c-1729846b3d25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:32.497675 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e067a0-3350-480d-ab2c-1729846b3d25 service swift] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:32.507442 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1181/2388] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Jul 12 22:09:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 22:09:34.708468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa88344-a24b-47a8-b184-6f5512b9aa8e None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:34.709992 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-1fa88344-a24b-47a8-b184-6f5512b9aa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:34.764058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:09:34.800591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa88344-a24b-47a8-b184-6f5512b9aa8e None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:34.805546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24866cc-3c61-432c-8b80-9b9af8752cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:34.806541 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24866cc-3c61-432c-8b80-9b9af8752cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:34.806891 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24866cc-3c61-432c-8b80-9b9af8752cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:34.807244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24866cc-3c61-432c-8b80-9b9af8752cb9 admin admin] PATH_INFO: `/v3/users/dc1d7070c0db4eb4bba5e1e018da5bad` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:34.808549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24866cc-3c61-432c-8b80-9b9af8752cb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc1d7070c0db4eb4bba5e1e018da5bad)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:34.810692 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24866cc-3c61-432c-8b80-9b9af8752cb9 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:34.868762 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d24866cc-3c61-432c-8b80-9b9af8752cb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc1d7070c0db4eb4bba5e1e018da5bad'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:34.884913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d24866cc-3c61-432c-8b80-9b9af8752cb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc1d7070c0db4eb4bba5e1e018da5bad'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:34.894669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d24866cc-3c61-432c-8b80-9b9af8752cb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc1d7070c0db4eb4bba5e1e018da5bad'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:34.906262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d24866cc-3c61-432c-8b80-9b9af8752cb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc1d7070c0db4eb4bba5e1e018da5bad'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:34.907334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d24866cc-3c61-432c-8b80-9b9af8752cb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc1d7070c0db4eb4bba5e1e018da5bad'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:34.917998 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1208/2389] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:09:34 2022] DELETE /identity/v3/users/dc1d7070c0db4eb4bba5e1e018da5bad => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:09:34.923360 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98041e1-69b5-4042-b59e-82d9cfcc7cce None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:34.924438 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98041e1-69b5-4042-b59e-82d9cfcc7cce None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:34.924906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98041e1-69b5-4042-b59e-82d9cfcc7cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:34.949264 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-e98041e1-69b5-4042-b59e-82d9cfcc7cce None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:09:34.950837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e98041e1-69b5-4042-b59e-82d9cfcc7cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:34.964805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:09:35.028514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b9a94-3776-411b-8761-8ca532c4dc95 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:35.030127 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b9a94-3776-411b-8761-8ca532c4dc95 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:35.030364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b9a94-3776-411b-8761-8ca532c4dc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:35.040813 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1182/2390] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:09:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:09:35.047395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98041e1-69b5-4042-b59e-82d9cfcc7cce None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:35.049744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e98041e1-69b5-4042-b59e-82d9cfcc7cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:35.050650 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-ba7b9a94-3776-411b-8761-8ca532c4dc95 None None] MFA Rules not processed for user `85dea53a4a034f5981b638bf92f2cf3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:09:35.051840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ba7b9a94-3776-411b-8761-8ca532c4dc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:35.112815 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1209/2391] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:09:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:09:35.117257 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98041e1-69b5-4042-b59e-82d9cfcc7cce None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:35.130566 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82edb1e2-c760-4d02-b2ec-05d8d92acbed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:35.131089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edb1e2-c760-4d02-b2ec-05d8d92acbed service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:35.131477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edb1e2-c760-4d02-b2ec-05d8d92acbed service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:35.131871 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edb1e2-c760-4d02-b2ec-05d8d92acbed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:35.144212 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82edb1e2-c760-4d02-b2ec-05d8d92acbed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:35.145755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82edb1e2-c760-4d02-b2ec-05d8d92acbed service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:35.156371 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1183/2392] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:09:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:09:35.690274 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b9a94-3776-411b-8761-8ca532c4dc95 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:09:35.692225 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ba7b9a94-3776-411b-8761-8ca532c4dc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:09:35.741354 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b9a94-3776-411b-8761-8ca532c4dc95 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:09:35.747141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:09:35.748120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:09:35.748290 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:09:35.748917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26 admin admin] PATH_INFO: `/v3/projects/f86df0806e8747159530bd66cd36e7f3` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:09:35.749895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f86df0806e8747159530bd66cd36e7f3)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:09:35.753529 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:09:35.833689 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f86df0806e8747159530bd66cd36e7f3', 'request_id': 'req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:35.840939 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f86df0806e8747159530bd66cd36e7f3', 'request_id': 'req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:35.845682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:09:35.865466 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f86df0806e8747159530bd66cd36e7f3', 'request_id': 'req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:35.882815 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26 admin admin] The token cache is being invalidate because project f86df0806e8747159530bd66cd36e7f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:09:35.883114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:35.883839 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:09:35.890499 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1210/2393] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:09:35 2022] DELETE /identity/v3/projects/f86df0806e8747159530bd66cd36e7f3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:09:45.152077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=45 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:09:45.875313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=4 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:10:00.698836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa23941-8e5e-4eaa-a20b-c54c52d997e0 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:00.699409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa23941-8e5e-4eaa-a20b-c54c52d997e0 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:00.699409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa23941-8e5e-4eaa-a20b-c54c52d997e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:00.716870 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-5aa23941-8e5e-4eaa-a20b-c54c52d997e0 None None] MFA Rules not processed for user `4a746b42824c4d13a1d0b0f2c5831259`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:10:00.718099 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-5aa23941-8e5e-4eaa-a20b-c54c52d997e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:00.777338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1184/2394] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:10:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:10:10.770404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=25 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:10:26.782642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:26.784410 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:26.865712 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa4a10-c6dd-4e68-a22b-3f9165ad9a26 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:26.871001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d4b2f-6be1-4abe-87a7-3b1c7bf87571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:26.871927 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d4b2f-6be1-4abe-87a7-3b1c7bf87571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:26.872227 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d4b2f-6be1-4abe-87a7-3b1c7bf87571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:26.872500 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d4b2f-6be1-4abe-87a7-3b1c7bf87571 admin admin] PATH_INFO: `/v3/users/4c137f9412de4914b77732e1ac29ec36` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:26.873784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d4b2f-6be1-4abe-87a7-3b1c7bf87571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c137f9412de4914b77732e1ac29ec36)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:26.875830 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d4b2f-6be1-4abe-87a7-3b1c7bf87571 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:26.933219 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-377d4b2f-6be1-4abe-87a7-3b1c7bf87571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c137f9412de4914b77732e1ac29ec36'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:10:26.948235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-377d4b2f-6be1-4abe-87a7-3b1c7bf87571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c137f9412de4914b77732e1ac29ec36'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:10:26.960758 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-377d4b2f-6be1-4abe-87a7-3b1c7bf87571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c137f9412de4914b77732e1ac29ec36'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:10:26.969400 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-377d4b2f-6be1-4abe-87a7-3b1c7bf87571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c137f9412de4914b77732e1ac29ec36'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:10:26.972013 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-377d4b2f-6be1-4abe-87a7-3b1c7bf87571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c137f9412de4914b77732e1ac29ec36'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:10:26.979525 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1211/2395] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:10:26 2022] DELETE /identity/v3/users/4c137f9412de4914b77732e1ac29ec36 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:10:27.190731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa23941-8e5e-4eaa-a20b-c54c52d997e0 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:27.202958 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa23941-8e5e-4eaa-a20b-c54c52d997e0 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:27.262852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3315b40-34d6-437b-8944-56d121de96f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:27.263755 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3315b40-34d6-437b-8944-56d121de96f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:27.264042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3315b40-34d6-437b-8944-56d121de96f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:27.264322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3315b40-34d6-437b-8944-56d121de96f9 admin admin] PATH_INFO: `/v3/projects/39f88660bffb4d17ba8b5a7a00c3f6d3` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:27.265577 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3315b40-34d6-437b-8944-56d121de96f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39f88660bffb4d17ba8b5a7a00c3f6d3)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:27.267636 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3315b40-34d6-437b-8944-56d121de96f9 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:27.318593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b3315b40-34d6-437b-8944-56d121de96f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39f88660bffb4d17ba8b5a7a00c3f6d3', 'request_id': 'req-b3315b40-34d6-437b-8944-56d121de96f9'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:10:27.325183 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b3315b40-34d6-437b-8944-56d121de96f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39f88660bffb4d17ba8b5a7a00c3f6d3', 'request_id': 'req-b3315b40-34d6-437b-8944-56d121de96f9'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:10:27.333433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b3315b40-34d6-437b-8944-56d121de96f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39f88660bffb4d17ba8b5a7a00c3f6d3', 'request_id': 'req-b3315b40-34d6-437b-8944-56d121de96f9'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:10:27.344426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b3315b40-34d6-437b-8944-56d121de96f9 admin admin] The token cache is being invalidate because project 39f88660bffb4d17ba8b5a7a00c3f6d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:10:27.344732 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b3315b40-34d6-437b-8944-56d121de96f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:10:27.345947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b3315b40-34d6-437b-8944-56d121de96f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:10:27.347573 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1185/2396] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:10:27 2022] DELETE /identity/v3/projects/39f88660bffb4d17ba8b5a7a00c3f6d3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:10:27.352659 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfb335-8ce4-4a53-acc9-e26b2ff5c172 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:27.353037 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfb335-8ce4-4a53-acc9-e26b2ff5c172 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:27.353324 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfb335-8ce4-4a53-acc9-e26b2ff5c172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:27.371696 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-bbcfb335-8ce4-4a53-acc9-e26b2ff5c172 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:10:27.372934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-bbcfb335-8ce4-4a53-acc9-e26b2ff5c172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:27.434597 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1212/2397] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:10:27.442993 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4125afc-3d64-484e-9d3b-f44759dbddd2 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:27.443297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4125afc-3d64-484e-9d3b-f44759dbddd2 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:27.443567 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4125afc-3d64-484e-9d3b-f44759dbddd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:27.462996 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-c4125afc-3d64-484e-9d3b-f44759dbddd2 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:10:27.464230 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c4125afc-3d64-484e-9d3b-f44759dbddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:27.472939 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1186/2398] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:10:27.477230 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfb335-8ce4-4a53-acc9-e26b2ff5c172 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:27.487725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfb335-8ce4-4a53-acc9-e26b2ff5c172 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:27.492543 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9399f49-d8a1-4f44-b6fe-6931f846c615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:27.493225 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9399f49-d8a1-4f44-b6fe-6931f846c615 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:27.493542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9399f49-d8a1-4f44-b6fe-6931f846c615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:27.493827 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9399f49-d8a1-4f44-b6fe-6931f846c615 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:27.494558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9399f49-d8a1-4f44-b6fe-6931f846c615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:10:27.494869 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9399f49-d8a1-4f44-b6fe-6931f846c615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:27.496484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9399f49-d8a1-4f44-b6fe-6931f846c615 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:27.508736 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1213/2399] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:10:27.513033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4125afc-3d64-484e-9d3b-f44759dbddd2 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:27.522792 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4125afc-3d64-484e-9d3b-f44759dbddd2 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:27.527655 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a9e08-03ca-47c7-9c51-1e255d5d8286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:27.528300 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a9e08-03ca-47c7-9c51-1e255d5d8286 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:27.528582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a9e08-03ca-47c7-9c51-1e255d5d8286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:27.528888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a9e08-03ca-47c7-9c51-1e255d5d8286 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:27.529574 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a9e08-03ca-47c7-9c51-1e255d5d8286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:27.531502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a9e08-03ca-47c7-9c51-1e255d5d8286 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:27.558407 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1187/2400] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:10:27 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:10:27.562640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9399f49-d8a1-4f44-b6fe-6931f846c615 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:27.572975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9399f49-d8a1-4f44-b6fe-6931f846c615 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:27.576826 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:10:27.631895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c8be1-e754-4352-953a-bf6a0e6b112b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:27.632565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c8be1-e754-4352-953a-bf6a0e6b112b admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:27.632890 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c8be1-e754-4352-953a-bf6a0e6b112b admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:27.633174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c8be1-e754-4352-953a-bf6a0e6b112b admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:27.633865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c8be1-e754-4352-953a-bf6a0e6b112b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:27.635802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c8be1-e754-4352-953a-bf6a0e6b112b admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:27.655515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1214/2401] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:10:27 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:10:27.660418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a9e08-03ca-47c7-9c51-1e255d5d8286 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:27.671855 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a9e08-03ca-47c7-9c51-1e255d5d8286 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:27.677841 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc44a0-05a6-4525-9adb-d33d1aa8b2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:27.678547 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc44a0-05a6-4525-9adb-d33d1aa8b2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:27.678840 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc44a0-05a6-4525-9adb-d33d1aa8b2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:27.679106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc44a0-05a6-4525-9adb-d33d1aa8b2c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:27.679840 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc44a0-05a6-4525-9adb-d33d1aa8b2c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:27.681488 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc44a0-05a6-4525-9adb-d33d1aa8b2c4 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:27.692462 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1188/2402] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:10:27.697060 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c8be1-e754-4352-953a-bf6a0e6b112b admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:27.707678 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c8be1-e754-4352-953a-bf6a0e6b112b admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:27.712231 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a60dd-7256-4532-a07a-62a7cade4221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:27.712919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a60dd-7256-4532-a07a-62a7cade4221 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:27.713201 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a60dd-7256-4532-a07a-62a7cade4221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:27.713478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a60dd-7256-4532-a07a-62a7cade4221 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:27.714203 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a60dd-7256-4532-a07a-62a7cade4221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:27.715783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a60dd-7256-4532-a07a-62a7cade4221 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:27.726299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1215/2403] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:10:27.730482 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc44a0-05a6-4525-9adb-d33d1aa8b2c4 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:27.740823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc44a0-05a6-4525-9adb-d33d1aa8b2c4 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:27.745337 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35e4f6-b37d-4a08-abab-a84150251509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:27.746201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35e4f6-b37d-4a08-abab-a84150251509 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:27.746482 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35e4f6-b37d-4a08-abab-a84150251509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:27.746763 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35e4f6-b37d-4a08-abab-a84150251509 admin admin] PATH_INFO: `/v3/projects/80e867d86756459f9e4f53a4238d393d/users/24a41c09f2e4487f9897c9888a0d752f/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:27.761662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35e4f6-b37d-4a08-abab-a84150251509 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e867d86756459f9e4f53a4238d393d, user_id=24a41c09f2e4487f9897c9888a0d752f, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:27.764465 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35e4f6-b37d-4a08-abab-a84150251509 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:27.777392 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1189/2404] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:10:27 2022] PUT /identity/v3/projects/80e867d86756459f9e4f53a4238d393d/users/24a41c09f2e4487f9897c9888a0d752f/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:10:27.783791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73570061-e2c6-432d-a252-f31a8f7d8739 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:27.784291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73570061-e2c6-432d-a252-f31a8f7d8739 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:27.784410 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73570061-e2c6-432d-a252-f31a8f7d8739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:27.802695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-73570061-e2c6-432d-a252-f31a8f7d8739 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:10:27.803898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-73570061-e2c6-432d-a252-f31a8f7d8739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:27.864117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1216/2405] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:10:27.873749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35e4f6-b37d-4a08-abab-a84150251509 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:27.875478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-fa35e4f6-b37d-4a08-abab-a84150251509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:27.907383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:10:27.935801 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35e4f6-b37d-4a08-abab-a84150251509 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:27.940011 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb0b5a-7971-44ed-9fb3-375bc61f8d73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:27.940655 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb0b5a-7971-44ed-9fb3-375bc61f8d73 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:27.940973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb0b5a-7971-44ed-9fb3-375bc61f8d73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:27.941252 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb0b5a-7971-44ed-9fb3-375bc61f8d73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:27.950616 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb0b5a-7971-44ed-9fb3-375bc61f8d73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:27.952048 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb0b5a-7971-44ed-9fb3-375bc61f8d73 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:27.963056 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1190/2406] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:10:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:10:32.045763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dab29-091c-4085-97d7-d9995119c419 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:32.046221 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dab29-091c-4085-97d7-d9995119c419 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:32.046410 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dab29-091c-4085-97d7-d9995119c419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:32.089296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-ca4dab29-091c-4085-97d7-d9995119c419 None None] MFA Rules not processed for user `24a41c09f2e4487f9897c9888a0d752f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:10:32.090936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ca4dab29-091c-4085-97d7-d9995119c419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:32.178106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb0b5a-7971-44ed-9fb3-375bc61f8d73 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:32.179663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1217/2407] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:10:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:10:32.188399 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb0b5a-7971-44ed-9fb3-375bc61f8d73 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:32.196444 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15723dfc-1615-49a5-b320-5e780fc2f67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:32.198027 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15723dfc-1615-49a5-b320-5e780fc2f67b admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:32.198027 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15723dfc-1615-49a5-b320-5e780fc2f67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:32.198285 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15723dfc-1615-49a5-b320-5e780fc2f67b admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:32.199064 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15723dfc-1615-49a5-b320-5e780fc2f67b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:32.201237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15723dfc-1615-49a5-b320-5e780fc2f67b admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:32.245679 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1191/2408] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:10:32 2022] POST /identity/v3/projects => generated 379 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:10:32.249388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4dab29-091c-4085-97d7-d9995119c419 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:32.281273 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4dab29-091c-4085-97d7-d9995119c419 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:32.297536 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:10:32.357926 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a318e19-7a41-4e01-b35b-bb72727fe31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:32.358608 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a318e19-7a41-4e01-b35b-bb72727fe31c admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:32.358894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a318e19-7a41-4e01-b35b-bb72727fe31c admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:32.359170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a318e19-7a41-4e01-b35b-bb72727fe31c admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:32.359906 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a318e19-7a41-4e01-b35b-bb72727fe31c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:32.362712 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a318e19-7a41-4e01-b35b-bb72727fe31c admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:32.388103 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1218/2409] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:10:32 2022] POST /identity/v3/users => generated 296 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:10:32.392412 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15723dfc-1615-49a5-b320-5e780fc2f67b admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:32.403473 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15723dfc-1615-49a5-b320-5e780fc2f67b admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:32.409281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4d961-5c1f-4e69-8c3e-db4024783ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:32.410084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d961-5c1f-4e69-8c3e-db4024783ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:32.410377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d961-5c1f-4e69-8c3e-db4024783ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:32.412791 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d961-5c1f-4e69-8c3e-db4024783ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:32.412791 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4d961-5c1f-4e69-8c3e-db4024783ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:32.414821 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4d961-5c1f-4e69-8c3e-db4024783ef3 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:32.427058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1192/2410] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:10:32.433260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a318e19-7a41-4e01-b35b-bb72727fe31c admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:32.444068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a318e19-7a41-4e01-b35b-bb72727fe31c admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:32.449240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b234aa3-bfc8-4cf4-8a19-3099b7819312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:32.450142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b234aa3-bfc8-4cf4-8a19-3099b7819312 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:32.451002 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b234aa3-bfc8-4cf4-8a19-3099b7819312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:32.451288 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b234aa3-bfc8-4cf4-8a19-3099b7819312 admin admin] PATH_INFO: `/v3/projects/840b8dfce15e451294dc402954ee5c6d/users/8bd938cbe36b4f0db2d3253a291963bb/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:32.467578 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b234aa3-bfc8-4cf4-8a19-3099b7819312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=840b8dfce15e451294dc402954ee5c6d, user_id=8bd938cbe36b4f0db2d3253a291963bb, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:32.470841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b234aa3-bfc8-4cf4-8a19-3099b7819312 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:32.488007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1219/2411] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:10:32 2022] PUT /identity/v3/projects/840b8dfce15e451294dc402954ee5c6d/users/8bd938cbe36b4f0db2d3253a291963bb/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:10:32.490766 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4d961-5c1f-4e69-8c3e-db4024783ef3 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:32.501633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4d961-5c1f-4e69-8c3e-db4024783ef3 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:32.562273 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962012b-e5a5-469e-a8af-211306e3b41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:32.563108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962012b-e5a5-469e-a8af-211306e3b41a admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:32.563458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962012b-e5a5-469e-a8af-211306e3b41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:32.563744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962012b-e5a5-469e-a8af-211306e3b41a admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:32.564475 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962012b-e5a5-469e-a8af-211306e3b41a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:32.566148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962012b-e5a5-469e-a8af-211306e3b41a admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:32.577278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1193/2412] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:10:32.581958 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b234aa3-bfc8-4cf4-8a19-3099b7819312 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:32.592808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b234aa3-bfc8-4cf4-8a19-3099b7819312 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:32.598302 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f452907-2076-4cdb-a820-94f2dbfdd9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:32.599284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f452907-2076-4cdb-a820-94f2dbfdd9ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:32.599627 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f452907-2076-4cdb-a820-94f2dbfdd9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:32.599969 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f452907-2076-4cdb-a820-94f2dbfdd9ac admin admin] PATH_INFO: `/v3/projects/840b8dfce15e451294dc402954ee5c6d/users/8bd938cbe36b4f0db2d3253a291963bb/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:32.602069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f452907-2076-4cdb-a820-94f2dbfdd9ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=840b8dfce15e451294dc402954ee5c6d, user_id=8bd938cbe36b4f0db2d3253a291963bb, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:32.604760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f452907-2076-4cdb-a820-94f2dbfdd9ac admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:32.616979 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1220/2413] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:10:32 2022] PUT /identity/v3/projects/840b8dfce15e451294dc402954ee5c6d/users/8bd938cbe36b4f0db2d3253a291963bb/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:10:37.446181 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508720f-8ad7-4f44-b1f8-9d2c54971342 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:37.446645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508720f-8ad7-4f44-b1f8-9d2c54971342 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:37.446935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508720f-8ad7-4f44-b1f8-9d2c54971342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:37.475520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-d508720f-8ad7-4f44-b1f8-9d2c54971342 None None] MFA Rules not processed for user `8bd938cbe36b4f0db2d3253a291963bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:10:37.477018 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d508720f-8ad7-4f44-b1f8-9d2c54971342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:37.553615 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1194/2414] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:10:37 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:10:37.568506 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f452907-2076-4cdb-a820-94f2dbfdd9ac admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:37.570626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5f452907-2076-4cdb-a820-94f2dbfdd9ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:37.661086 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f452907-2076-4cdb-a820-94f2dbfdd9ac admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:37.667235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e14b82-9f11-4cd9-862f-9fb36047d020 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:37.667933 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e14b82-9f11-4cd9-862f-9fb36047d020 service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:37.668231 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e14b82-9f11-4cd9-862f-9fb36047d020 service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:37.668513 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e14b82-9f11-4cd9-862f-9fb36047d020 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:37.728996 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e14b82-9f11-4cd9-862f-9fb36047d020 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:37.730631 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e14b82-9f11-4cd9-862f-9fb36047d020 service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:37.740089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1221/2415] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:10:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:10:40.844756 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508720f-8ad7-4f44-b1f8-9d2c54971342 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:40.869987 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:10:40.918932 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508720f-8ad7-4f44-b1f8-9d2c54971342 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:40.922792 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f6f18-6e0f-4359-90aa-d892ae7ed688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:40.923718 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f6f18-6e0f-4359-90aa-d892ae7ed688 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:40.924006 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f6f18-6e0f-4359-90aa-d892ae7ed688 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:40.924311 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f6f18-6e0f-4359-90aa-d892ae7ed688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:40.926153 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c80f6f18-6e0f-4359-90aa-d892ae7ed688 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:40.942770 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f6f18-6e0f-4359-90aa-d892ae7ed688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:40.944829 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f6f18-6e0f-4359-90aa-d892ae7ed688 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:40.957980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1195/2416] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:10:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:10:41.099062 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02e8e9-12e6-4186-a729-f569fe3392a0 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:41.099545 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02e8e9-12e6-4186-a729-f569fe3392a0 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:41.099684 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02e8e9-12e6-4186-a729-f569fe3392a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:41.106911 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:10:41.120691 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-dd02e8e9-12e6-4186-a729-f569fe3392a0 None None] MFA Rules not processed for user `309de655c8d64223a9c2d75ff5618411`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:10:41.124821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-dd02e8e9-12e6-4186-a729-f569fe3392a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:41.192082 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1222/2417] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:10:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:10:41.257364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f6f18-6e0f-4359-90aa-d892ae7ed688 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:41.259066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c80f6f18-6e0f-4359-90aa-d892ae7ed688 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:41.334031 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f6f18-6e0f-4359-90aa-d892ae7ed688 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:41.343127 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07a665-1674-4373-a665-1a37176e5961 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:41.344298 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07a665-1674-4373-a665-1a37176e5961 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:41.345144 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07a665-1674-4373-a665-1a37176e5961 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:41.345497 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07a665-1674-4373-a665-1a37176e5961 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:41.348545 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1d07a665-1674-4373-a665-1a37176e5961 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:41.365913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07a665-1674-4373-a665-1a37176e5961 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:41.367371 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07a665-1674-4373-a665-1a37176e5961 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:41.382066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1196/2418] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:10:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:10:51.184644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=23 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:10:51.376970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=44 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:10:54.440018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f291b-7a49-4245-a2c8-d48f1a6e0761 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:54.440609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f291b-7a49-4245-a2c8-d48f1a6e0761 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:54.440609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f291b-7a49-4245-a2c8-d48f1a6e0761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:54.459291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-487f291b-7a49-4245-a2c8-d48f1a6e0761 None None] MFA Rules not processed for user `24a41c09f2e4487f9897c9888a0d752f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:10:54.460564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-487f291b-7a49-4245-a2c8-d48f1a6e0761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:54.468072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1223/2419] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:10:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:10:55.152037 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a7f76-5e81-428d-ae47-0669a04d3b14 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:55.152579 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a7f76-5e81-428d-ae47-0669a04d3b14 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:55.153348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a7f76-5e81-428d-ae47-0669a04d3b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:55.171800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-b34a7f76-5e81-428d-ae47-0669a04d3b14 None None] MFA Rules not processed for user `85dea53a4a034f5981b638bf92f2cf3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:10:55.174111 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b34a7f76-5e81-428d-ae47-0669a04d3b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:55.231502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1197/2420] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:10:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:10:55.243193 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f291b-7a49-4245-a2c8-d48f1a6e0761 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:55.244688 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-487f291b-7a49-4245-a2c8-d48f1a6e0761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:55.260559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f291b-7a49-4245-a2c8-d48f1a6e0761 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:55.266525 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f9a14-12fd-4b79-b685-bda46394a4c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:55.267240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f9a14-12fd-4b79-b685-bda46394a4c6 service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:55.267522 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f9a14-12fd-4b79-b685-bda46394a4c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:55.267794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f9a14-12fd-4b79-b685-bda46394a4c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:55.269594 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d09f9a14-12fd-4b79-b685-bda46394a4c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:55.280662 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f9a14-12fd-4b79-b685-bda46394a4c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:55.283081 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f9a14-12fd-4b79-b685-bda46394a4c6 service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:55.292592 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1224/2421] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:10:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:10:57.257300 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a7f76-5e81-428d-ae47-0669a04d3b14 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:10:57.272780 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a7f76-5e81-428d-ae47-0669a04d3b14 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:10:57.278770 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f1cd7-bf19-40bf-95a5-71faf518580d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:10:57.279425 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f1cd7-bf19-40bf-95a5-71faf518580d service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:10:57.279706 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f1cd7-bf19-40bf-95a5-71faf518580d service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:10:57.279978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f1cd7-bf19-40bf-95a5-71faf518580d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:10:57.283016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-560f1cd7-bf19-40bf-95a5-71faf518580d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:10:57.355069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f1cd7-bf19-40bf-95a5-71faf518580d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:10:57.356730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f1cd7-bf19-40bf-95a5-71faf518580d service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:10:57.368822 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1198/2422] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:10:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:11:05.288199 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=9 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:11:05.329172 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8fbf6-2c5a-4ddd-93b0-eeb518b97cdd None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:05.329581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8fbf6-2c5a-4ddd-93b0-eeb518b97cdd None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:05.329972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8fbf6-2c5a-4ddd-93b0-eeb518b97cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:05.349630 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-8da8fbf6-2c5a-4ddd-93b0-eeb518b97cdd None None] MFA Rules not processed for user `8d6b823cc7dc434986f77739fa0259ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:11:05.350859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8da8fbf6-2c5a-4ddd-93b0-eeb518b97cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:05.408125 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1225/2423] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Jul 12 22:11:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:11:05.457871 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f1cd7-bf19-40bf-95a5-71faf518580d service nova] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:05.459372 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-560f1cd7-bf19-40bf-95a5-71faf518580d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:05.528259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f1cd7-bf19-40bf-95a5-71faf518580d service nova] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:05.533790 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd2c15-cc0e-4883-95b2-bc375e1005b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837bb5ac18554dfdbbcc776689639494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:05.534433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd2c15-cc0e-4883-95b2-bc375e1005b5 service swift] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:05.534717 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd2c15-cc0e-4883-95b2-bc375e1005b5 service swift] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:05.534998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd2c15-cc0e-4883-95b2-bc375e1005b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:05.543229 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd2c15-cc0e-4883-95b2-bc375e1005b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:05.544849 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd2c15-cc0e-4883-95b2-bc375e1005b5 service swift] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:05.553619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1199/2424] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Jul 12 22:11:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 22:11:11.511604 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8fbf6-2c5a-4ddd-93b0-eeb518b97cdd None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:11.532678 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8fbf6-2c5a-4ddd-93b0-eeb518b97cdd None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:11.532678 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db70bb-0cf9-491e-afd4-8ea0d5ba7369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:11.532678 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db70bb-0cf9-491e-afd4-8ea0d5ba7369 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:11.533625 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db70bb-0cf9-491e-afd4-8ea0d5ba7369 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:11.534410 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db70bb-0cf9-491e-afd4-8ea0d5ba7369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:11.536482 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b4db70bb-0cf9-491e-afd4-8ea0d5ba7369 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:11.548664 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db70bb-0cf9-491e-afd4-8ea0d5ba7369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:11.550890 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db70bb-0cf9-491e-afd4-8ea0d5ba7369 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:11.572099 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1226/2425] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:11:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:11:14.826205 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd2c15-cc0e-4883-95b2-bc375e1005b5 service swift] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:14.837597 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd2c15-cc0e-4883-95b2-bc375e1005b5 service swift] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:14.841998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cc0e2-9168-4571-bef6-8a8390f9bcf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:14.842901 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cc0e2-9168-4571-bef6-8a8390f9bcf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:14.843536 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cc0e2-9168-4571-bef6-8a8390f9bcf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:14.844028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cc0e2-9168-4571-bef6-8a8390f9bcf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:14.854201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cc0e2-9168-4571-bef6-8a8390f9bcf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:14.855968 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cc0e2-9168-4571-bef6-8a8390f9bcf3 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:14.867611 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1200/2426] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:11:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:11:19.947572 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db70bb-0cf9-491e-afd4-8ea0d5ba7369 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:19.949232 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b4db70bb-0cf9-491e-afd4-8ea0d5ba7369 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:20.024241 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db70bb-0cf9-491e-afd4-8ea0d5ba7369 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:20.029286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3eed8-097d-406e-a8ed-c6556f1a5c13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f1ef52f8eb42a59a50a69873b9ffaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:20.029966 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3eed8-097d-406e-a8ed-c6556f1a5c13 service placement] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:20.030842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3eed8-097d-406e-a8ed-c6556f1a5c13 service placement] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:20.031126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3eed8-097d-406e-a8ed-c6556f1a5c13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:20.032974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a1a3eed8-097d-406e-a8ed-c6556f1a5c13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:20.044643 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3eed8-097d-406e-a8ed-c6556f1a5c13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:20.046356 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3eed8-097d-406e-a8ed-c6556f1a5c13 service placement] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:20.056176 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1227/2427] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:11:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:11:22.024325 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceec5c0-4970-4144-85c2-9b55ad152b2b None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:22.025130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceec5c0-4970-4144-85c2-9b55ad152b2b None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:22.025130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceec5c0-4970-4144-85c2-9b55ad152b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:22.031502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=92 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:11:22.044876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-dceec5c0-4970-4144-85c2-9b55ad152b2b None None] MFA Rules not processed for user `8d6b823cc7dc434986f77739fa0259ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:11:22.047035 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-dceec5c0-4970-4144-85c2-9b55ad152b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:22.054866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1201/2428] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Jul 12 22:11:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:11:22.067778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3eed8-097d-406e-a8ed-c6556f1a5c13 service placement] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:22.080277 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3eed8-097d-406e-a8ed-c6556f1a5c13 service placement] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:22.085480 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b38a23-8677-4b35-a6d6-2a954c16593b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837bb5ac18554dfdbbcc776689639494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:22.086069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b38a23-8677-4b35-a6d6-2a954c16593b service swift] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:22.086353 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b38a23-8677-4b35-a6d6-2a954c16593b service swift] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:22.086632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b38a23-8677-4b35-a6d6-2a954c16593b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:22.096040 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b38a23-8677-4b35-a6d6-2a954c16593b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:22.097642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b38a23-8677-4b35-a6d6-2a954c16593b service swift] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:22.106098 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1228/2429] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Jul 12 22:11:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 22:11:32.039304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:11:32.102469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=69 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:11:36.049094 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceec5c0-4970-4144-85c2-9b55ad152b2b None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:36.066489 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceec5c0-4970-4144-85c2-9b55ad152b2b None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:36.073302 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdefe9-af89-42b1-8aba-5fcfd21ebaea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:36.074047 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdefe9-af89-42b1-8aba-5fcfd21ebaea service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:36.074402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdefe9-af89-42b1-8aba-5fcfd21ebaea service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:36.074784 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdefe9-af89-42b1-8aba-5fcfd21ebaea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:36.076670 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-7efdefe9-af89-42b1-8aba-5fcfd21ebaea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:36.089913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdefe9-af89-42b1-8aba-5fcfd21ebaea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:36.091641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdefe9-af89-42b1-8aba-5fcfd21ebaea service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:36.107973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1202/2430] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:11:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:11:36.938050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b38a23-8677-4b35-a6d6-2a954c16593b service swift] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:36.950466 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b38a23-8677-4b35-a6d6-2a954c16593b service swift] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:36.956652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13bf48-561a-4d5e-9be2-e0828ebff8a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:36.961984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13bf48-561a-4d5e-9be2-e0828ebff8a3 service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:36.961984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13bf48-561a-4d5e-9be2-e0828ebff8a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:36.961984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13bf48-561a-4d5e-9be2-e0828ebff8a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:36.961984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ea13bf48-561a-4d5e-9be2-e0828ebff8a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:36.973643 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13bf48-561a-4d5e-9be2-e0828ebff8a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:36.975467 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13bf48-561a-4d5e-9be2-e0828ebff8a3 service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:36.988886 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1229/2431] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:11:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:11:38.121222 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fb548-bfe3-411d-b657-dcf841f55687 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:38.121726 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fb548-bfe3-411d-b657-dcf841f55687 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:38.121809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fb548-bfe3-411d-b657-dcf841f55687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:38.142531 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-dc9fb548-bfe3-411d-b657-dcf841f55687 None None] MFA Rules not processed for user `8d6b823cc7dc434986f77739fa0259ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:11:38.143909 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-dc9fb548-bfe3-411d-b657-dcf841f55687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:38.151762 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1203/2432] 10.222.0.44 () {58 vars in 967 bytes} [Tue Jul 12 22:11:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:11:38.166984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13bf48-561a-4d5e-9be2-e0828ebff8a3 service nova] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:38.183534 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13bf48-561a-4d5e-9be2-e0828ebff8a3 service nova] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:38.192508 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461536fc-1cf5-426f-a90b-1546f977e11e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837bb5ac18554dfdbbcc776689639494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:38.193673 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461536fc-1cf5-426f-a90b-1546f977e11e service swift] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:38.194173 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461536fc-1cf5-426f-a90b-1546f977e11e service swift] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:38.194656 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461536fc-1cf5-426f-a90b-1546f977e11e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:38.206572 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461536fc-1cf5-426f-a90b-1546f977e11e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:38.208905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461536fc-1cf5-426f-a90b-1546f977e11e service swift] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:38.225018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1230/2433] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Jul 12 22:11:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 22:11:48.137471 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=9 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:11:48.215110 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=12 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:11:53.245467 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fb548-bfe3-411d-b657-dcf841f55687 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:53.258895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fb548-bfe3-411d-b657-dcf841f55687 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:53.263929 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628c31b-758d-4189-87df-9a9ee11cb9eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f1ef52f8eb42a59a50a69873b9ffaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:53.264691 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628c31b-758d-4189-87df-9a9ee11cb9eb service placement] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:53.264994 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628c31b-758d-4189-87df-9a9ee11cb9eb service placement] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:53.265276 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628c31b-758d-4189-87df-9a9ee11cb9eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:53.267501 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-9628c31b-758d-4189-87df-9a9ee11cb9eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:53.280235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628c31b-758d-4189-87df-9a9ee11cb9eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:53.282422 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628c31b-758d-4189-87df-9a9ee11cb9eb service placement] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:53.293439 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1204/2434] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:11:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:11:53.307028 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461536fc-1cf5-426f-a90b-1546f977e11e service swift] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:53.308638 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-461536fc-1cf5-426f-a90b-1546f977e11e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:53.380242 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461536fc-1cf5-426f-a90b-1546f977e11e service swift] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:53.384826 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73eafb-db5e-4659-ad21-370a2043daae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:53.385697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73eafb-db5e-4659-ad21-370a2043daae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:53.385983 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73eafb-db5e-4659-ad21-370a2043daae admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:53.386262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73eafb-db5e-4659-ad21-370a2043daae admin admin] PATH_INFO: `/v3/users/309de655c8d64223a9c2d75ff5618411` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:53.387450 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73eafb-db5e-4659-ad21-370a2043daae admin admin] RBAC: Authorizing `identity:delete_user(user_id=309de655c8d64223a9c2d75ff5618411)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:53.389469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb73eafb-db5e-4659-ad21-370a2043daae admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:53.435972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-eb73eafb-db5e-4659-ad21-370a2043daae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '309de655c8d64223a9c2d75ff5618411'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:53.447851 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-eb73eafb-db5e-4659-ad21-370a2043daae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '309de655c8d64223a9c2d75ff5618411'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:53.454902 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-eb73eafb-db5e-4659-ad21-370a2043daae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '309de655c8d64223a9c2d75ff5618411'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:53.461948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-eb73eafb-db5e-4659-ad21-370a2043daae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '309de655c8d64223a9c2d75ff5618411'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:53.462561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-eb73eafb-db5e-4659-ad21-370a2043daae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '309de655c8d64223a9c2d75ff5618411'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:53.469295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1231/2435] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:11:53 2022] DELETE /identity/v3/users/309de655c8d64223a9c2d75ff5618411 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:11:53.654865 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628c31b-758d-4189-87df-9a9ee11cb9eb service placement] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:53.665490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628c31b-758d-4189-87df-9a9ee11cb9eb service placement] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:53.722669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb7271-f374-467f-9534-4d7e920d1e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:53.723621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb7271-f374-467f-9534-4d7e920d1e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:53.723846 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb7271-f374-467f-9534-4d7e920d1e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:53.724127 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb7271-f374-467f-9534-4d7e920d1e58 admin admin] PATH_INFO: `/v3/projects/4b4f6be5087f4fd18a60969372480425` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:53.725335 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb7271-f374-467f-9534-4d7e920d1e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b4f6be5087f4fd18a60969372480425)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:53.727308 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb7271-f374-467f-9534-4d7e920d1e58 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:53.775116 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9fbb7271-f374-467f-9534-4d7e920d1e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b4f6be5087f4fd18a60969372480425', 'request_id': 'req-9fbb7271-f374-467f-9534-4d7e920d1e58'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:53.781171 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9fbb7271-f374-467f-9534-4d7e920d1e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b4f6be5087f4fd18a60969372480425', 'request_id': 'req-9fbb7271-f374-467f-9534-4d7e920d1e58'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:53.788389 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9fbb7271-f374-467f-9534-4d7e920d1e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b4f6be5087f4fd18a60969372480425', 'request_id': 'req-9fbb7271-f374-467f-9534-4d7e920d1e58'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:53.799305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9fbb7271-f374-467f-9534-4d7e920d1e58 admin admin] The token cache is being invalidate because project 4b4f6be5087f4fd18a60969372480425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:11:53.799648 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9fbb7271-f374-467f-9534-4d7e920d1e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:53.800394 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9fbb7271-f374-467f-9534-4d7e920d1e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:53.802555 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1205/2436] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:11:53 2022] DELETE /identity/v3/projects/4b4f6be5087f4fd18a60969372480425 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:11:53.807675 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950536c-e046-40fe-960d-337b37820944 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:53.807972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950536c-e046-40fe-960d-337b37820944 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:53.808288 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950536c-e046-40fe-960d-337b37820944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:53.827183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-0950536c-e046-40fe-960d-337b37820944 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:11:53.828436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0950536c-e046-40fe-960d-337b37820944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:53.903142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1232/2437] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:11:53.912222 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d9c64-cae6-48a8-88a3-674894a2eedd None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:53.912543 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d9c64-cae6-48a8-88a3-674894a2eedd None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:53.912926 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d9c64-cae6-48a8-88a3-674894a2eedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:53.932200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-cf0d9c64-cae6-48a8-88a3-674894a2eedd None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:11:53.933515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-cf0d9c64-cae6-48a8-88a3-674894a2eedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:53.942238 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1206/2438] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:11:53.946740 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950536c-e046-40fe-960d-337b37820944 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:53.956902 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950536c-e046-40fe-960d-337b37820944 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:53.961586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3774c4-6593-405e-8849-d6d2763c7b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:53.962254 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3774c4-6593-405e-8849-d6d2763c7b72 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:53.962542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3774c4-6593-405e-8849-d6d2763c7b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:53.962823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3774c4-6593-405e-8849-d6d2763c7b72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:53.963559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3774c4-6593-405e-8849-d6d2763c7b72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:11:53.963871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3774c4-6593-405e-8849-d6d2763c7b72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:53.965545 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3774c4-6593-405e-8849-d6d2763c7b72 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:53.977781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1233/2439] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:11:53.982097 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d9c64-cae6-48a8-88a3-674894a2eedd None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:53.992287 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d9c64-cae6-48a8-88a3-674894a2eedd None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:53.997078 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c552430-1ea3-46b5-aeed-c93c0c7df720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:53.997731 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c552430-1ea3-46b5-aeed-c93c0c7df720 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:53.998018 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c552430-1ea3-46b5-aeed-c93c0c7df720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:53.998847 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c552430-1ea3-46b5-aeed-c93c0c7df720 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:53.999545 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c552430-1ea3-46b5-aeed-c93c0c7df720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:54.001499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c552430-1ea3-46b5-aeed-c93c0c7df720 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:54.026452 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1207/2440] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:11:53 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:11:54.030780 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3774c4-6593-405e-8849-d6d2763c7b72 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:54.040911 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3774c4-6593-405e-8849-d6d2763c7b72 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:54.044414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:11:54.099713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a1fd1-5c46-409a-b0e9-c81b47e5bf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:54.100383 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a1fd1-5c46-409a-b0e9-c81b47e5bf12 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:54.100663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a1fd1-5c46-409a-b0e9-c81b47e5bf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:54.100970 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a1fd1-5c46-409a-b0e9-c81b47e5bf12 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:54.101654 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a1fd1-5c46-409a-b0e9-c81b47e5bf12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:54.103581 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a1fd1-5c46-409a-b0e9-c81b47e5bf12 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:54.125469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1234/2441] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:11:54 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:11:54.129767 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c552430-1ea3-46b5-aeed-c93c0c7df720 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:54.140786 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c552430-1ea3-46b5-aeed-c93c0c7df720 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:54.145522 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51261b9-89c3-4036-84bf-250301c4798c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:54.146185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51261b9-89c3-4036-84bf-250301c4798c admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:54.146473 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51261b9-89c3-4036-84bf-250301c4798c admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:54.146745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51261b9-89c3-4036-84bf-250301c4798c admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:54.147468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51261b9-89c3-4036-84bf-250301c4798c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:54.149124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51261b9-89c3-4036-84bf-250301c4798c admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:54.161183 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1208/2442] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:11:54.165563 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a1fd1-5c46-409a-b0e9-c81b47e5bf12 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:54.177758 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a1fd1-5c46-409a-b0e9-c81b47e5bf12 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:54.182759 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c207a09-06c0-4787-9d17-b8f64b0c7954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:54.183609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c207a09-06c0-4787-9d17-b8f64b0c7954 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:54.183927 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c207a09-06c0-4787-9d17-b8f64b0c7954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:54.184207 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c207a09-06c0-4787-9d17-b8f64b0c7954 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:54.185065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c207a09-06c0-4787-9d17-b8f64b0c7954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:54.187018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c207a09-06c0-4787-9d17-b8f64b0c7954 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:54.201696 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1235/2443] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:11:54.207714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51261b9-89c3-4036-84bf-250301c4798c admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:54.224681 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51261b9-89c3-4036-84bf-250301c4798c admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:54.230096 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672ca71-53ea-4c9a-8953-2b5f335ba052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:54.231375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672ca71-53ea-4c9a-8953-2b5f335ba052 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:54.231688 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672ca71-53ea-4c9a-8953-2b5f335ba052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:54.232051 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672ca71-53ea-4c9a-8953-2b5f335ba052 admin admin] PATH_INFO: `/v3/projects/88e9c1bc655149ac8f3f744f39820daf/users/e4e59d879d2a44c19f49c35784857c18/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:54.256044 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672ca71-53ea-4c9a-8953-2b5f335ba052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e9c1bc655149ac8f3f744f39820daf, user_id=e4e59d879d2a44c19f49c35784857c18, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:54.260490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672ca71-53ea-4c9a-8953-2b5f335ba052 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:54.275863 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1209/2444] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:11:54 2022] PUT /identity/v3/projects/88e9c1bc655149ac8f3f744f39820daf/users/e4e59d879d2a44c19f49c35784857c18/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:11:54.287705 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9577a-9169-47ba-b141-29ef9bc57b96 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:54.287999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9577a-9169-47ba-b141-29ef9bc57b96 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:54.288286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9577a-9169-47ba-b141-29ef9bc57b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:54.316912 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-97b9577a-9169-47ba-b141-29ef9bc57b96 None None] MFA Rules not processed for user `e4e59d879d2a44c19f49c35784857c18`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:11:54.318144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-97b9577a-9169-47ba-b141-29ef9bc57b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:54.382942 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1236/2445] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:11:54 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:11:54.387422 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672ca71-53ea-4c9a-8953-2b5f335ba052 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:54.400498 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:11:54.453852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672ca71-53ea-4c9a-8953-2b5f335ba052 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:54.459030 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91970c-2524-4d45-8321-d5b5b4914098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:54.459693 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91970c-2524-4d45-8321-d5b5b4914098 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:54.459978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91970c-2524-4d45-8321-d5b5b4914098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:54.460254 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91970c-2524-4d45-8321-d5b5b4914098 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:54.460988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91970c-2524-4d45-8321-d5b5b4914098 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:54.462933 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91970c-2524-4d45-8321-d5b5b4914098 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:54.489964 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1210/2446] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:11:54 2022] POST /identity/v3/projects => generated 363 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:11:54.494027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9577a-9169-47ba-b141-29ef9bc57b96 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:54.505026 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9577a-9169-47ba-b141-29ef9bc57b96 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:54.565021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7fbcf-153e-400d-ac1f-4aac714a26c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:54.565514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7fbcf-153e-400d-ac1f-4aac714a26c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:54.565808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7fbcf-153e-400d-ac1f-4aac714a26c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:54.566094 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7fbcf-153e-400d-ac1f-4aac714a26c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:54.566804 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7fbcf-153e-400d-ac1f-4aac714a26c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:54.569122 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7fbcf-153e-400d-ac1f-4aac714a26c8 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:54.596978 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1237/2447] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:11:54 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:11:54.598369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91970c-2524-4d45-8321-d5b5b4914098 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:54.609278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91970c-2524-4d45-8321-d5b5b4914098 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:54.614661 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993d2d0-2e7a-4f54-a3cc-567be0f88bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:54.615335 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993d2d0-2e7a-4f54-a3cc-567be0f88bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:54.615621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993d2d0-2e7a-4f54-a3cc-567be0f88bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:54.615901 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993d2d0-2e7a-4f54-a3cc-567be0f88bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:54.616629 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993d2d0-2e7a-4f54-a3cc-567be0f88bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:54.618272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993d2d0-2e7a-4f54-a3cc-567be0f88bb0 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:54.629091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1211/2448] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:11:54.633530 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7fbcf-153e-400d-ac1f-4aac714a26c8 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:54.640519 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:11:54.644404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7fbcf-153e-400d-ac1f-4aac714a26c8 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:54.650546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43dc13-26b4-4963-8d80-0f261ee147e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:54.651451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43dc13-26b4-4963-8d80-0f261ee147e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:54.651735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43dc13-26b4-4963-8d80-0f261ee147e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:54.652013 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43dc13-26b4-4963-8d80-0f261ee147e2 admin admin] PATH_INFO: `/v3/projects/5f005adeb9d047568ba9caef303b5805/users/a950ca3c404b4c8aacd39553715de33a/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:54.667542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43dc13-26b4-4963-8d80-0f261ee147e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f005adeb9d047568ba9caef303b5805, user_id=a950ca3c404b4c8aacd39553715de33a, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:54.670452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43dc13-26b4-4963-8d80-0f261ee147e2 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:54.683102 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1238/2449] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:11:54 2022] PUT /identity/v3/projects/5f005adeb9d047568ba9caef303b5805/users/a950ca3c404b4c8aacd39553715de33a/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:11:54.687174 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993d2d0-2e7a-4f54-a3cc-567be0f88bb0 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:54.697442 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993d2d0-2e7a-4f54-a3cc-567be0f88bb0 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:54.754836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a73cb-bc2c-40d8-ae4c-33b22cbf7d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:54.755422 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a73cb-bc2c-40d8-ae4c-33b22cbf7d64 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:54.755708 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a73cb-bc2c-40d8-ae4c-33b22cbf7d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:54.755985 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a73cb-bc2c-40d8-ae4c-33b22cbf7d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:54.756702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a73cb-bc2c-40d8-ae4c-33b22cbf7d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:54.758351 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a73cb-bc2c-40d8-ae4c-33b22cbf7d64 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:54.768837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1212/2450] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:11:54.773045 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43dc13-26b4-4963-8d80-0f261ee147e2 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:54.783901 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43dc13-26b4-4963-8d80-0f261ee147e2 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:54.788436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7a728-32eb-42fa-8c9e-d91ff8fe53c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:54.789335 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7a728-32eb-42fa-8c9e-d91ff8fe53c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:54.789613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7a728-32eb-42fa-8c9e-d91ff8fe53c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:54.789883 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7a728-32eb-42fa-8c9e-d91ff8fe53c1 admin admin] PATH_INFO: `/v3/projects/5f005adeb9d047568ba9caef303b5805/users/a950ca3c404b4c8aacd39553715de33a/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:54.791856 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7a728-32eb-42fa-8c9e-d91ff8fe53c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f005adeb9d047568ba9caef303b5805, user_id=a950ca3c404b4c8aacd39553715de33a, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:54.794510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7a728-32eb-42fa-8c9e-d91ff8fe53c1 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:54.805299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1239/2451] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:11:54 2022] PUT /identity/v3/projects/5f005adeb9d047568ba9caef303b5805/users/a950ca3c404b4c8aacd39553715de33a/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:11:54.815833 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907375b-b99b-4ce5-9952-082347a4b542 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:54.816117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907375b-b99b-4ce5-9952-082347a4b542 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:54.816394 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907375b-b99b-4ce5-9952-082347a4b542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:54.844836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-d907375b-b99b-4ce5-9952-082347a4b542 None None] MFA Rules not processed for user `a950ca3c404b4c8aacd39553715de33a`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:11:54.846093 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d907375b-b99b-4ce5-9952-082347a4b542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:54.918858 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1213/2452] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:11:54 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:11:54.930709 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7a728-32eb-42fa-8c9e-d91ff8fe53c1 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:54.932446 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-f0d7a728-32eb-42fa-8c9e-d91ff8fe53c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:54.990405 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7a728-32eb-42fa-8c9e-d91ff8fe53c1 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:54.994159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf120b1-a1a8-454c-b87a-7d28674946bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:54.994795 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf120b1-a1a8-454c-b87a-7d28674946bf service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:54.995080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf120b1-a1a8-454c-b87a-7d28674946bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:54.995384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf120b1-a1a8-454c-b87a-7d28674946bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:55.005480 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf120b1-a1a8-454c-b87a-7d28674946bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:55.006787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf120b1-a1a8-454c-b87a-7d28674946bf service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:55.016997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1240/2453] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:11:54 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:11:55.427225 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907375b-b99b-4ce5-9952-082347a4b542 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:55.437274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:11:55.491614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907375b-b99b-4ce5-9952-082347a4b542 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:55.496154 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44c979-b733-4695-8e55-61c64ec93e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:55.497052 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44c979-b733-4695-8e55-61c64ec93e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:55.497334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44c979-b733-4695-8e55-61c64ec93e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:55.497609 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44c979-b733-4695-8e55-61c64ec93e24 admin admin] PATH_INFO: `/v3/users/e4e59d879d2a44c19f49c35784857c18` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:55.499060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44c979-b733-4695-8e55-61c64ec93e24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4e59d879d2a44c19f49c35784857c18)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:55.501047 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44c979-b733-4695-8e55-61c64ec93e24 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:55.548089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ab44c979-b733-4695-8e55-61c64ec93e24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4e59d879d2a44c19f49c35784857c18'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:55.555259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ab44c979-b733-4695-8e55-61c64ec93e24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4e59d879d2a44c19f49c35784857c18'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:55.562750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ab44c979-b733-4695-8e55-61c64ec93e24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4e59d879d2a44c19f49c35784857c18'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:55.563426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ab44c979-b733-4695-8e55-61c64ec93e24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4e59d879d2a44c19f49c35784857c18'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:55.564028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ab44c979-b733-4695-8e55-61c64ec93e24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4e59d879d2a44c19f49c35784857c18'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:55.582188 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1214/2454] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:11:55 2022] DELETE /identity/v3/users/e4e59d879d2a44c19f49c35784857c18 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:11:55.586654 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217310-d829-4ac8-af05-975d6913db0f None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:55.587139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217310-d829-4ac8-af05-975d6913db0f None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:55.587251 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217310-d829-4ac8-af05-975d6913db0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:55.605414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-87217310-d829-4ac8-af05-975d6913db0f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:11:55.606615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-87217310-d829-4ac8-af05-975d6913db0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:55.665965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1241/2455] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:11:55.675542 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44c979-b733-4695-8e55-61c64ec93e24 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:55.676995 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ab44c979-b733-4695-8e55-61c64ec93e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:55.743234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44c979-b733-4695-8e55-61c64ec93e24 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:55.747361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75166be-1577-40f5-8947-d612687d19d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:55.748674 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75166be-1577-40f5-8947-d612687d19d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:55.749052 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75166be-1577-40f5-8947-d612687d19d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:55.749566 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75166be-1577-40f5-8947-d612687d19d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:55.759959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75166be-1577-40f5-8947-d612687d19d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:55.761687 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75166be-1577-40f5-8947-d612687d19d9 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:55.774058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1215/2456] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:11:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:11:56.055841 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87217310-d829-4ac8-af05-975d6913db0f None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:56.057551 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-87217310-d829-4ac8-af05-975d6913db0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:56.071817 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87217310-d829-4ac8-af05-975d6913db0f None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:56.077239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52153ddc-977d-4938-ad6b-f43eebd5d619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:56.078108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52153ddc-977d-4938-ad6b-f43eebd5d619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:56.078481 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52153ddc-977d-4938-ad6b-f43eebd5d619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:56.078659 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52153ddc-977d-4938-ad6b-f43eebd5d619 admin admin] PATH_INFO: `/v3/projects/88e9c1bc655149ac8f3f744f39820daf` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:56.080001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52153ddc-977d-4938-ad6b-f43eebd5d619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e9c1bc655149ac8f3f744f39820daf)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:56.081982 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52153ddc-977d-4938-ad6b-f43eebd5d619 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:56.131392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-52153ddc-977d-4938-ad6b-f43eebd5d619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e9c1bc655149ac8f3f744f39820daf', 'request_id': 'req-52153ddc-977d-4938-ad6b-f43eebd5d619'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:56.136860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-52153ddc-977d-4938-ad6b-f43eebd5d619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e9c1bc655149ac8f3f744f39820daf', 'request_id': 'req-52153ddc-977d-4938-ad6b-f43eebd5d619'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:56.144899 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-52153ddc-977d-4938-ad6b-f43eebd5d619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e9c1bc655149ac8f3f744f39820daf', 'request_id': 'req-52153ddc-977d-4938-ad6b-f43eebd5d619'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:56.146067 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:11:56.159115 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-52153ddc-977d-4938-ad6b-f43eebd5d619 admin admin] The token cache is being invalidate because project 88e9c1bc655149ac8f3f744f39820daf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:11:56.159407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-52153ddc-977d-4938-ad6b-f43eebd5d619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:56.160064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-52153ddc-977d-4938-ad6b-f43eebd5d619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:56.161679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1242/2457] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:11:56 2022] DELETE /identity/v3/projects/88e9c1bc655149ac8f3f744f39820daf => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:11:56.165923 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75166be-1577-40f5-8947-d612687d19d9 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:56.176251 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75166be-1577-40f5-8947-d612687d19d9 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:56.205892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:11:56.235265 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f1662-1f91-4650-a43a-d8ed6de7d8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:56.236145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f1662-1f91-4650-a43a-d8ed6de7d8b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:56.236430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f1662-1f91-4650-a43a-d8ed6de7d8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:56.236725 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f1662-1f91-4650-a43a-d8ed6de7d8b6 admin admin] PATH_INFO: `/v3/users/a950ca3c404b4c8aacd39553715de33a` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:56.237944 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f1662-1f91-4650-a43a-d8ed6de7d8b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a950ca3c404b4c8aacd39553715de33a)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:56.239887 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f1662-1f91-4650-a43a-d8ed6de7d8b6 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:56.295738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-d74f1662-1f91-4650-a43a-d8ed6de7d8b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a950ca3c404b4c8aacd39553715de33a'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:56.305213 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-d74f1662-1f91-4650-a43a-d8ed6de7d8b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a950ca3c404b4c8aacd39553715de33a'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:56.314107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-d74f1662-1f91-4650-a43a-d8ed6de7d8b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a950ca3c404b4c8aacd39553715de33a'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:56.314721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-d74f1662-1f91-4650-a43a-d8ed6de7d8b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a950ca3c404b4c8aacd39553715de33a'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:56.315325 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-d74f1662-1f91-4650-a43a-d8ed6de7d8b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a950ca3c404b4c8aacd39553715de33a'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:56.332916 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1216/2458] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:11:56 2022] DELETE /identity/v3/users/a950ca3c404b4c8aacd39553715de33a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:11:56.613561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52153ddc-977d-4938-ad6b-f43eebd5d619 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:56.615740 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-52153ddc-977d-4938-ad6b-f43eebd5d619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:56.636884 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52153ddc-977d-4938-ad6b-f43eebd5d619 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:56.702264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136b208-9374-4b05-a0f4-fb4fb4f768f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:56.703130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136b208-9374-4b05-a0f4-fb4fb4f768f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:56.703415 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136b208-9374-4b05-a0f4-fb4fb4f768f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:56.703826 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136b208-9374-4b05-a0f4-fb4fb4f768f9 admin admin] PATH_INFO: `/v3/projects/5f005adeb9d047568ba9caef303b5805` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:56.705037 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136b208-9374-4b05-a0f4-fb4fb4f768f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f005adeb9d047568ba9caef303b5805)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:56.706888 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136b208-9374-4b05-a0f4-fb4fb4f768f9 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:56.758607 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d136b208-9374-4b05-a0f4-fb4fb4f768f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f005adeb9d047568ba9caef303b5805', 'request_id': 'req-d136b208-9374-4b05-a0f4-fb4fb4f768f9'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:56.764757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d136b208-9374-4b05-a0f4-fb4fb4f768f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f005adeb9d047568ba9caef303b5805', 'request_id': 'req-d136b208-9374-4b05-a0f4-fb4fb4f768f9'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:56.775077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d136b208-9374-4b05-a0f4-fb4fb4f768f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f005adeb9d047568ba9caef303b5805', 'request_id': 'req-d136b208-9374-4b05-a0f4-fb4fb4f768f9'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:56.788425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d136b208-9374-4b05-a0f4-fb4fb4f768f9 admin admin] The token cache is being invalidate because project 5f005adeb9d047568ba9caef303b5805 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:11:56.788842 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d136b208-9374-4b05-a0f4-fb4fb4f768f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:56.789506 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d136b208-9374-4b05-a0f4-fb4fb4f768f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:11:56.791072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1243/2459] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:11:56 2022] DELETE /identity/v3/projects/5f005adeb9d047568ba9caef303b5805 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:11:56.796103 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f53ac-f324-4f85-b025-9b08a7a902d0 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:56.796592 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f53ac-f324-4f85-b025-9b08a7a902d0 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:56.796688 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f53ac-f324-4f85-b025-9b08a7a902d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:56.816019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-848f53ac-f324-4f85-b025-9b08a7a902d0 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:11:56.817445 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-848f53ac-f324-4f85-b025-9b08a7a902d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:56.881835 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1217/2460] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:11:56.889853 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6275085-ae0d-4354-aadf-816ddc9c506b None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:56.890150 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6275085-ae0d-4354-aadf-816ddc9c506b None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:56.890438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6275085-ae0d-4354-aadf-816ddc9c506b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:56.908764 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-b6275085-ae0d-4354-aadf-816ddc9c506b None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:11:56.909980 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b6275085-ae0d-4354-aadf-816ddc9c506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:56.918445 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1244/2461] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:11:56.923329 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f53ac-f324-4f85-b025-9b08a7a902d0 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:56.933341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f53ac-f324-4f85-b025-9b08a7a902d0 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:56.938929 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411dfed-fc91-4afb-b414-e36555216a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:56.939604 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411dfed-fc91-4afb-b414-e36555216a8e admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:56.939980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411dfed-fc91-4afb-b414-e36555216a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:56.940321 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411dfed-fc91-4afb-b414-e36555216a8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:56.941165 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411dfed-fc91-4afb-b414-e36555216a8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:11:56.941691 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411dfed-fc91-4afb-b414-e36555216a8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:56.943547 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411dfed-fc91-4afb-b414-e36555216a8e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:56.955577 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1218/2462] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:11:56.960224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6275085-ae0d-4354-aadf-816ddc9c506b None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:56.970476 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6275085-ae0d-4354-aadf-816ddc9c506b None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:56.975277 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0218d-a588-4d57-870d-12357d4b5eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:56.976110 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0218d-a588-4d57-870d-12357d4b5eef admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:56.976396 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0218d-a588-4d57-870d-12357d4b5eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:56.976678 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0218d-a588-4d57-870d-12357d4b5eef admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:56.977427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0218d-a588-4d57-870d-12357d4b5eef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:56.979322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0218d-a588-4d57-870d-12357d4b5eef admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:57.010967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1245/2463] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:11:56 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:11:57.015401 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411dfed-fc91-4afb-b414-e36555216a8e admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:57.025959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411dfed-fc91-4afb-b414-e36555216a8e admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:57.069780 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:11:57.097640 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ae567-5bee-48a7-a3d3-bef9543cc590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:57.098878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ae567-5bee-48a7-a3d3-bef9543cc590 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:57.098878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ae567-5bee-48a7-a3d3-bef9543cc590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:57.099133 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ae567-5bee-48a7-a3d3-bef9543cc590 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:57.100353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ae567-5bee-48a7-a3d3-bef9543cc590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:57.102078 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ae567-5bee-48a7-a3d3-bef9543cc590 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:57.128155 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1219/2464] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:11:57 2022] POST /identity/v3/users => generated 282 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:11:57.132352 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0218d-a588-4d57-870d-12357d4b5eef admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:57.143559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0218d-a588-4d57-870d-12357d4b5eef admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:57.148886 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547643-8bdd-443c-b1ac-0dae40b73441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:57.149558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547643-8bdd-443c-b1ac-0dae40b73441 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:57.149847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547643-8bdd-443c-b1ac-0dae40b73441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:57.150129 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547643-8bdd-443c-b1ac-0dae40b73441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:57.150865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec547643-8bdd-443c-b1ac-0dae40b73441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:57.152499 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec547643-8bdd-443c-b1ac-0dae40b73441 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:57.165152 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1246/2465] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:11:57.169262 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ae567-5bee-48a7-a3d3-bef9543cc590 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:57.180523 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ae567-5bee-48a7-a3d3-bef9543cc590 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:57.185183 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f4447-bc72-494c-b802-c2ac90d582cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:57.185889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f4447-bc72-494c-b802-c2ac90d582cc admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:57.186273 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f4447-bc72-494c-b802-c2ac90d582cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:57.186558 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f4447-bc72-494c-b802-c2ac90d582cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:57.187288 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f4447-bc72-494c-b802-c2ac90d582cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:57.189073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f4447-bc72-494c-b802-c2ac90d582cc admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:57.199095 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1220/2466] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:11:57.203313 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547643-8bdd-443c-b1ac-0dae40b73441 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:57.213408 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547643-8bdd-443c-b1ac-0dae40b73441 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:57.219111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa6b98-b9ee-4b9e-8e63-65c34c01d579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:57.219988 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa6b98-b9ee-4b9e-8e63-65c34c01d579 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:57.220294 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa6b98-b9ee-4b9e-8e63-65c34c01d579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:57.220558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa6b98-b9ee-4b9e-8e63-65c34c01d579 admin admin] PATH_INFO: `/v3/projects/09bacc5fcc40469782dc9ecccc95f0ce/users/564f0a8ee5384719b3829e5ea5f11064/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:57.237732 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa6b98-b9ee-4b9e-8e63-65c34c01d579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09bacc5fcc40469782dc9ecccc95f0ce, user_id=564f0a8ee5384719b3829e5ea5f11064, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:57.240624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa6b98-b9ee-4b9e-8e63-65c34c01d579 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:57.254211 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1247/2467] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:11:57 2022] PUT /identity/v3/projects/09bacc5fcc40469782dc9ecccc95f0ce/users/564f0a8ee5384719b3829e5ea5f11064/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:11:57.265020 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed8203-cc62-4ade-b1d1-7e1ec847ebba None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:57.265314 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed8203-cc62-4ade-b1d1-7e1ec847ebba None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:57.265593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed8203-cc62-4ade-b1d1-7e1ec847ebba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:57.292167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-2fed8203-cc62-4ade-b1d1-7e1ec847ebba None None] MFA Rules not processed for user `564f0a8ee5384719b3829e5ea5f11064`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:11:57.293476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-2fed8203-cc62-4ade-b1d1-7e1ec847ebba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:57.365344 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1221/2468] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:11:57 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:11:57.369999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa6b98-b9ee-4b9e-8e63-65c34c01d579 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:57.391065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:11:57.434748 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa6b98-b9ee-4b9e-8e63-65c34c01d579 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:57.439820 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b69080-d71d-42af-a04d-36c0a3edf74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:57.440416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b69080-d71d-42af-a04d-36c0a3edf74a admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:57.440702 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b69080-d71d-42af-a04d-36c0a3edf74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:57.441017 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b69080-d71d-42af-a04d-36c0a3edf74a admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:57.441695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b69080-d71d-42af-a04d-36c0a3edf74a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:57.443864 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b69080-d71d-42af-a04d-36c0a3edf74a admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:57.470110 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1248/2469] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:11:57 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:11:57.474372 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed8203-cc62-4ade-b1d1-7e1ec847ebba None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:57.485676 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed8203-cc62-4ade-b1d1-7e1ec847ebba None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:57.549183 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6daba73-8e26-4404-8fc5-e09e8be56c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:57.549894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daba73-8e26-4404-8fc5-e09e8be56c96 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:57.550187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daba73-8e26-4404-8fc5-e09e8be56c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:57.550493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daba73-8e26-4404-8fc5-e09e8be56c96 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:57.551201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6daba73-8e26-4404-8fc5-e09e8be56c96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:57.553293 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6daba73-8e26-4404-8fc5-e09e8be56c96 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:57.574517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1222/2470] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:11:57 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:11:57.579165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b69080-d71d-42af-a04d-36c0a3edf74a admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:57.589809 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b69080-d71d-42af-a04d-36c0a3edf74a admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:57.596382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8802fe-689b-49b6-8076-b5e00686c572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:57.597178 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8802fe-689b-49b6-8076-b5e00686c572 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:57.597467 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8802fe-689b-49b6-8076-b5e00686c572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:57.597756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8802fe-689b-49b6-8076-b5e00686c572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:57.598484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8802fe-689b-49b6-8076-b5e00686c572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:57.600097 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8802fe-689b-49b6-8076-b5e00686c572 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:57.611625 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1249/2471] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:11:57.615876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6daba73-8e26-4404-8fc5-e09e8be56c96 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:57.626196 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6daba73-8e26-4404-8fc5-e09e8be56c96 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:57.630990 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f8ce4-2486-4162-a11d-64819fafd7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:57.631876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f8ce4-2486-4162-a11d-64819fafd7a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:57.632169 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f8ce4-2486-4162-a11d-64819fafd7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:57.632451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f8ce4-2486-4162-a11d-64819fafd7a1 admin admin] PATH_INFO: `/v3/projects/6aa5a2cca2cd428aa5e165e6c7fa19b1/users/babc1108a7ae4584b1f73bbf6a64f056/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:57.648193 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f8ce4-2486-4162-a11d-64819fafd7a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aa5a2cca2cd428aa5e165e6c7fa19b1, user_id=babc1108a7ae4584b1f73bbf6a64f056, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:57.650906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f8ce4-2486-4162-a11d-64819fafd7a1 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:57.665296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1223/2472] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:11:57 2022] PUT /identity/v3/projects/6aa5a2cca2cd428aa5e165e6c7fa19b1/users/babc1108a7ae4584b1f73bbf6a64f056/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:11:57.669398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8802fe-689b-49b6-8076-b5e00686c572 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:57.679642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8802fe-689b-49b6-8076-b5e00686c572 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:57.744362 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175c4ae-de6c-4102-a751-d176538b9c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:57.744984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175c4ae-de6c-4102-a751-d176538b9c94 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:57.745274 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175c4ae-de6c-4102-a751-d176538b9c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:57.745554 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175c4ae-de6c-4102-a751-d176538b9c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:57.746278 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175c4ae-de6c-4102-a751-d176538b9c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:57.747887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175c4ae-de6c-4102-a751-d176538b9c94 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:57.759022 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1250/2473] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:11:57.763313 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f8ce4-2486-4162-a11d-64819fafd7a1 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:57.775282 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f8ce4-2486-4162-a11d-64819fafd7a1 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:57.780650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b8688-ff17-4ba7-b918-ac380313687d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:57.781587 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b8688-ff17-4ba7-b918-ac380313687d admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:57.781896 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b8688-ff17-4ba7-b918-ac380313687d admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:57.782177 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b8688-ff17-4ba7-b918-ac380313687d admin admin] PATH_INFO: `/v3/projects/6aa5a2cca2cd428aa5e165e6c7fa19b1/users/babc1108a7ae4584b1f73bbf6a64f056/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:57.784201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b8688-ff17-4ba7-b918-ac380313687d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aa5a2cca2cd428aa5e165e6c7fa19b1, user_id=babc1108a7ae4584b1f73bbf6a64f056, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:57.786892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b8688-ff17-4ba7-b918-ac380313687d admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:57.793690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:11:57.801303 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1224/2474] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:11:57 2022] PUT /identity/v3/projects/6aa5a2cca2cd428aa5e165e6c7fa19b1/users/babc1108a7ae4584b1f73bbf6a64f056/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:11:57.813761 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451435d5-93c0-42f7-80de-1543360b4b1f None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:57.814054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451435d5-93c0-42f7-80de-1543360b4b1f None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:57.814340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451435d5-93c0-42f7-80de-1543360b4b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:57.843108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-451435d5-93c0-42f7-80de-1543360b4b1f None None] MFA Rules not processed for user `babc1108a7ae4584b1f73bbf6a64f056`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:11:57.844345 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-451435d5-93c0-42f7-80de-1543360b4b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:57.917421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1251/2475] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:11:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:11:57.928827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b8688-ff17-4ba7-b918-ac380313687d admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:57.931140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-f10b8688-ff17-4ba7-b918-ac380313687d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:11:57.991253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b8688-ff17-4ba7-b918-ac380313687d admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:57.995206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc5284-f34d-4ef9-b4c7-09576f64d94a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:57.995780 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc5284-f34d-4ef9-b4c7-09576f64d94a service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:57.996058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc5284-f34d-4ef9-b4c7-09576f64d94a service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:57.996350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc5284-f34d-4ef9-b4c7-09576f64d94a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:58.005910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc5284-f34d-4ef9-b4c7-09576f64d94a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:58.007375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc5284-f34d-4ef9-b4c7-09576f64d94a service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:58.017765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1225/2476] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:11:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:11:58.093578 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451435d5-93c0-42f7-80de-1543360b4b1f None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:11:58.103198 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451435d5-93c0-42f7-80de-1543360b4b1f None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:11:58.106915 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef0f72-fa09-4b98-9c80-d0fbe58581d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:11:58.107523 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef0f72-fa09-4b98-9c80-d0fbe58581d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:11:58.107798 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef0f72-fa09-4b98-9c80-d0fbe58581d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:11:58.108098 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef0f72-fa09-4b98-9c80-d0fbe58581d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:11:58.121829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:11:58.173265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef0f72-fa09-4b98-9c80-d0fbe58581d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:11:58.174700 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef0f72-fa09-4b98-9c80-d0fbe58581d0 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:11:58.184772 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1252/2477] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:11:58 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:07.340046 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc5284-f34d-4ef9-b4c7-09576f64d94a service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:07.341812 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c0fc5284-f34d-4ef9-b4c7-09576f64d94a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:07.405643 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc5284-f34d-4ef9-b4c7-09576f64d94a service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:07.409687 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190eb155-5f81-41ad-8560-d1ae8aed342b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:07.410481 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190eb155-5f81-41ad-8560-d1ae8aed342b service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:07.410774 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190eb155-5f81-41ad-8560-d1ae8aed342b service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:07.411055 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190eb155-5f81-41ad-8560-d1ae8aed342b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:07.412994 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-190eb155-5f81-41ad-8560-d1ae8aed342b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:07.484519 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190eb155-5f81-41ad-8560-d1ae8aed342b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:07.486053 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190eb155-5f81-41ad-8560-d1ae8aed342b service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:07.486843 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef0f72-fa09-4b98-9c80-d0fbe58581d0 service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:07.496102 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1226/2478] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:12:07.498672 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef0f72-fa09-4b98-9c80-d0fbe58581d0 service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:07.503302 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925e635-141d-4783-80ed-96d2f54ba866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:07.504177 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925e635-141d-4783-80ed-96d2f54ba866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:07.504471 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925e635-141d-4783-80ed-96d2f54ba866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:07.504772 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925e635-141d-4783-80ed-96d2f54ba866 admin admin] PATH_INFO: `/v3/users/564f0a8ee5384719b3829e5ea5f11064` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:07.505973 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925e635-141d-4783-80ed-96d2f54ba866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=564f0a8ee5384719b3829e5ea5f11064)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:07.507952 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925e635-141d-4783-80ed-96d2f54ba866 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:07.558822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-8925e635-141d-4783-80ed-96d2f54ba866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '564f0a8ee5384719b3829e5ea5f11064'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:07.571785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-8925e635-141d-4783-80ed-96d2f54ba866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '564f0a8ee5384719b3829e5ea5f11064'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:07.579654 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-8925e635-141d-4783-80ed-96d2f54ba866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '564f0a8ee5384719b3829e5ea5f11064'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:07.587951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-8925e635-141d-4783-80ed-96d2f54ba866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '564f0a8ee5384719b3829e5ea5f11064'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:07.588933 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-8925e635-141d-4783-80ed-96d2f54ba866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '564f0a8ee5384719b3829e5ea5f11064'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:07.595824 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1253/2479] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:12:07 2022] DELETE /identity/v3/users/564f0a8ee5384719b3829e5ea5f11064 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:07.600337 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66165002-c766-4339-a306-668877eeb302 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:07.600684 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66165002-c766-4339-a306-668877eeb302 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:07.601113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66165002-c766-4339-a306-668877eeb302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:07.620923 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-66165002-c766-4339-a306-668877eeb302 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:07.622193 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-66165002-c766-4339-a306-668877eeb302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:07.673614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:07.688281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1227/2480] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:07.701211 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925e635-141d-4783-80ed-96d2f54ba866 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:07.703056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8925e635-141d-4783-80ed-96d2f54ba866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:07.764456 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925e635-141d-4783-80ed-96d2f54ba866 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:07.768372 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f279caf-a35c-43f5-bb59-b30ad83ea435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:07.769168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f279caf-a35c-43f5-bb59-b30ad83ea435 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:07.769483 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f279caf-a35c-43f5-bb59-b30ad83ea435 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:07.769836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f279caf-a35c-43f5-bb59-b30ad83ea435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:07.780508 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f279caf-a35c-43f5-bb59-b30ad83ea435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:07.782016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f279caf-a35c-43f5-bb59-b30ad83ea435 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:07.792195 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1254/2481] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:08.078752 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66165002-c766-4339-a306-668877eeb302 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:08.080947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-66165002-c766-4339-a306-668877eeb302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:08.106440 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66165002-c766-4339-a306-668877eeb302 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:08.112089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d5832-4ee3-4700-8414-5718461e4d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:08.113060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d5832-4ee3-4700-8414-5718461e4d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:08.113323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d5832-4ee3-4700-8414-5718461e4d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:08.113606 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d5832-4ee3-4700-8414-5718461e4d0e admin admin] PATH_INFO: `/v3/projects/09bacc5fcc40469782dc9ecccc95f0ce` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:08.114812 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d5832-4ee3-4700-8414-5718461e4d0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=09bacc5fcc40469782dc9ecccc95f0ce)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:08.117288 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d5832-4ee3-4700-8414-5718461e4d0e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:08.169106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-213d5832-4ee3-4700-8414-5718461e4d0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09bacc5fcc40469782dc9ecccc95f0ce', 'request_id': 'req-213d5832-4ee3-4700-8414-5718461e4d0e'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:08.175644 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-213d5832-4ee3-4700-8414-5718461e4d0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09bacc5fcc40469782dc9ecccc95f0ce', 'request_id': 'req-213d5832-4ee3-4700-8414-5718461e4d0e'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:08.184070 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-213d5832-4ee3-4700-8414-5718461e4d0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09bacc5fcc40469782dc9ecccc95f0ce', 'request_id': 'req-213d5832-4ee3-4700-8414-5718461e4d0e'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:08.195498 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-213d5832-4ee3-4700-8414-5718461e4d0e admin admin] The token cache is being invalidate because project 09bacc5fcc40469782dc9ecccc95f0ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:12:08.195790 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-213d5832-4ee3-4700-8414-5718461e4d0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:08.196464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-213d5832-4ee3-4700-8414-5718461e4d0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:08.201426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1228/2482] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:12:08 2022] DELETE /identity/v3/projects/09bacc5fcc40469782dc9ecccc95f0ce => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:08.202844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f279caf-a35c-43f5-bb59-b30ad83ea435 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:08.213224 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f279caf-a35c-43f5-bb59-b30ad83ea435 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:08.241487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:08.279376 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2dd03f-6c81-47eb-8d16-9808e08643d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:08.280585 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dd03f-6c81-47eb-8d16-9808e08643d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:08.280964 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dd03f-6c81-47eb-8d16-9808e08643d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:08.281301 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dd03f-6c81-47eb-8d16-9808e08643d6 admin admin] PATH_INFO: `/v3/users/babc1108a7ae4584b1f73bbf6a64f056` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:08.282547 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2dd03f-6c81-47eb-8d16-9808e08643d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=babc1108a7ae4584b1f73bbf6a64f056)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:08.284513 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2dd03f-6c81-47eb-8d16-9808e08643d6 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:08.337626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ef2dd03f-6c81-47eb-8d16-9808e08643d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'babc1108a7ae4584b1f73bbf6a64f056'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:08.351054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ef2dd03f-6c81-47eb-8d16-9808e08643d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'babc1108a7ae4584b1f73bbf6a64f056'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:08.359961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ef2dd03f-6c81-47eb-8d16-9808e08643d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'babc1108a7ae4584b1f73bbf6a64f056'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:08.368415 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ef2dd03f-6c81-47eb-8d16-9808e08643d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'babc1108a7ae4584b1f73bbf6a64f056'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:08.369109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ef2dd03f-6c81-47eb-8d16-9808e08643d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'babc1108a7ae4584b1f73bbf6a64f056'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:08.377609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1255/2483] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:12:08 2022] DELETE /identity/v3/users/babc1108a7ae4584b1f73bbf6a64f056 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:08.670636 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d5832-4ee3-4700-8414-5718461e4d0e admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:08.672676 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-213d5832-4ee3-4700-8414-5718461e4d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:08.686642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d5832-4ee3-4700-8414-5718461e4d0e admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:08.762865 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887dfb0-eca5-4ae1-ba95-c87fa8f44aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:08.764403 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887dfb0-eca5-4ae1-ba95-c87fa8f44aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:08.764804 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887dfb0-eca5-4ae1-ba95-c87fa8f44aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:08.765149 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887dfb0-eca5-4ae1-ba95-c87fa8f44aa2 admin admin] PATH_INFO: `/v3/projects/6aa5a2cca2cd428aa5e165e6c7fa19b1` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:08.766519 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887dfb0-eca5-4ae1-ba95-c87fa8f44aa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aa5a2cca2cd428aa5e165e6c7fa19b1)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:08.769033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887dfb0-eca5-4ae1-ba95-c87fa8f44aa2 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:08.827740 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-7887dfb0-eca5-4ae1-ba95-c87fa8f44aa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aa5a2cca2cd428aa5e165e6c7fa19b1', 'request_id': 'req-7887dfb0-eca5-4ae1-ba95-c87fa8f44aa2'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:08.835647 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-7887dfb0-eca5-4ae1-ba95-c87fa8f44aa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aa5a2cca2cd428aa5e165e6c7fa19b1', 'request_id': 'req-7887dfb0-eca5-4ae1-ba95-c87fa8f44aa2'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:08.844584 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-7887dfb0-eca5-4ae1-ba95-c87fa8f44aa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aa5a2cca2cd428aa5e165e6c7fa19b1', 'request_id': 'req-7887dfb0-eca5-4ae1-ba95-c87fa8f44aa2'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:08.857799 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-7887dfb0-eca5-4ae1-ba95-c87fa8f44aa2 admin admin] The token cache is being invalidate because project 6aa5a2cca2cd428aa5e165e6c7fa19b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:12:08.858198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-7887dfb0-eca5-4ae1-ba95-c87fa8f44aa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:08.858777 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-7887dfb0-eca5-4ae1-ba95-c87fa8f44aa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:08.865744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1229/2484] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:12:08 2022] DELETE /identity/v3/projects/6aa5a2cca2cd428aa5e165e6c7fa19b1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:08.867542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf5436-8252-4415-855e-d72068d1d727 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:08.868018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf5436-8252-4415-855e-d72068d1d727 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:08.868115 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf5436-8252-4415-855e-d72068d1d727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:08.887675 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-81cf5436-8252-4415-855e-d72068d1d727 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:08.889532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-81cf5436-8252-4415-855e-d72068d1d727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:08.969860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1256/2485] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:08.973045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32c6c6-0a18-4d40-8b53-46cb167130b1 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:08.973045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32c6c6-0a18-4d40-8b53-46cb167130b1 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:08.973045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32c6c6-0a18-4d40-8b53-46cb167130b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:08.980425 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:08.992434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-0a32c6c6-0a18-4d40-8b53-46cb167130b1 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:08.994124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0a32c6c6-0a18-4d40-8b53-46cb167130b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:09.002278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1230/2486] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:09.006511 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf5436-8252-4415-855e-d72068d1d727 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:09.016801 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf5436-8252-4415-855e-d72068d1d727 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:09.021494 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9864498-fe79-47c3-ac12-3ed803968cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:09.022154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9864498-fe79-47c3-ac12-3ed803968cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:09.022438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9864498-fe79-47c3-ac12-3ed803968cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:09.022719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9864498-fe79-47c3-ac12-3ed803968cd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:09.023454 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9864498-fe79-47c3-ac12-3ed803968cd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:12:09.023764 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9864498-fe79-47c3-ac12-3ed803968cd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:09.025429 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9864498-fe79-47c3-ac12-3ed803968cd0 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:09.037866 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1257/2487] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:12:09.042240 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32c6c6-0a18-4d40-8b53-46cb167130b1 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:09.052140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32c6c6-0a18-4d40-8b53-46cb167130b1 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:09.057004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b428efa-fee0-435a-9fe9-78e65ff48503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:09.057668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b428efa-fee0-435a-9fe9-78e65ff48503 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:09.057950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b428efa-fee0-435a-9fe9-78e65ff48503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:09.058328 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b428efa-fee0-435a-9fe9-78e65ff48503 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:09.058939 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b428efa-fee0-435a-9fe9-78e65ff48503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:09.061161 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b428efa-fee0-435a-9fe9-78e65ff48503 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:09.092070 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1231/2488] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:12:09 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:09.093757 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9864498-fe79-47c3-ac12-3ed803968cd0 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:09.104083 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9864498-fe79-47c3-ac12-3ed803968cd0 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:09.138743 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:09.165810 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d213e86-d6a5-45e9-8897-d4d60e05f56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:09.166497 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d213e86-d6a5-45e9-8897-d4d60e05f56f admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:09.166785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d213e86-d6a5-45e9-8897-d4d60e05f56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:09.167070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d213e86-d6a5-45e9-8897-d4d60e05f56f admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:09.167761 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d213e86-d6a5-45e9-8897-d4d60e05f56f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:09.169836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d213e86-d6a5-45e9-8897-d4d60e05f56f admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:09.198729 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1258/2489] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:12:09 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:09.199959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b428efa-fee0-435a-9fe9-78e65ff48503 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:09.211631 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b428efa-fee0-435a-9fe9-78e65ff48503 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:09.217519 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64416b5-dd4d-4f1d-b0dd-78a779f3c78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:09.218191 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64416b5-dd4d-4f1d-b0dd-78a779f3c78a admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:09.218474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64416b5-dd4d-4f1d-b0dd-78a779f3c78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:09.218750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64416b5-dd4d-4f1d-b0dd-78a779f3c78a admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:09.219478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64416b5-dd4d-4f1d-b0dd-78a779f3c78a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:09.221142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64416b5-dd4d-4f1d-b0dd-78a779f3c78a admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:09.232945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1232/2490] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:09.237295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d213e86-d6a5-45e9-8897-d4d60e05f56f admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:09.257803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d213e86-d6a5-45e9-8897-d4d60e05f56f admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:09.263254 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab842f-0a01-4af9-8f57-765284f6f217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:09.264883 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab842f-0a01-4af9-8f57-765284f6f217 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:09.265392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab842f-0a01-4af9-8f57-765284f6f217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:09.265897 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab842f-0a01-4af9-8f57-765284f6f217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:09.267388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab842f-0a01-4af9-8f57-765284f6f217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:09.274656 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab842f-0a01-4af9-8f57-765284f6f217 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:09.289450 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1259/2491] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:09.294001 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64416b5-dd4d-4f1d-b0dd-78a779f3c78a admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:09.306519 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64416b5-dd4d-4f1d-b0dd-78a779f3c78a admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:09.312658 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced6030-da6d-4817-8c9a-d45e3fc8edac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:09.313700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced6030-da6d-4817-8c9a-d45e3fc8edac admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:09.314055 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced6030-da6d-4817-8c9a-d45e3fc8edac admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:09.314410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced6030-da6d-4817-8c9a-d45e3fc8edac admin admin] PATH_INFO: `/v3/projects/61da8c681703405a8c7dbbc5a6bc671e/users/47794b1db02c4f6882c7019608d8e50b/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:09.335019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced6030-da6d-4817-8c9a-d45e3fc8edac admin admin] RBAC: Authorizing `identity:create_grant(project_id=61da8c681703405a8c7dbbc5a6bc671e, user_id=47794b1db02c4f6882c7019608d8e50b, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:09.337921 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced6030-da6d-4817-8c9a-d45e3fc8edac admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:09.354754 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1233/2492] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:12:09 2022] PUT /identity/v3/projects/61da8c681703405a8c7dbbc5a6bc671e/users/47794b1db02c4f6882c7019608d8e50b/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:09.365145 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b4884-f80c-4127-93b6-cc84df1f18f1 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:09.365357 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b4884-f80c-4127-93b6-cc84df1f18f1 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:09.365597 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b4884-f80c-4127-93b6-cc84df1f18f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:09.396653 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-0f7b4884-f80c-4127-93b6-cc84df1f18f1 None None] MFA Rules not processed for user `47794b1db02c4f6882c7019608d8e50b`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:09.397905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0f7b4884-f80c-4127-93b6-cc84df1f18f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:09.470690 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1260/2493] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:09.477220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced6030-da6d-4817-8c9a-d45e3fc8edac admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:09.557228 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced6030-da6d-4817-8c9a-d45e3fc8edac admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:09.564953 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f6f56-89f4-4e9f-a8bb-1abfddc83e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:09.565667 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f6f56-89f4-4e9f-a8bb-1abfddc83e70 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:09.565959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f6f56-89f4-4e9f-a8bb-1abfddc83e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:09.566235 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f6f56-89f4-4e9f-a8bb-1abfddc83e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:09.567070 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f6f56-89f4-4e9f-a8bb-1abfddc83e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:09.569367 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f6f56-89f4-4e9f-a8bb-1abfddc83e70 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:09.602078 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1234/2494] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:12:09 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:09.603193 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b4884-f80c-4127-93b6-cc84df1f18f1 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:09.615613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b4884-f80c-4127-93b6-cc84df1f18f1 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:09.686153 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0cad1-4da8-4f9d-911e-7d7ec9ae5be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:09.686993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0cad1-4da8-4f9d-911e-7d7ec9ae5be6 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:09.687307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0cad1-4da8-4f9d-911e-7d7ec9ae5be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:09.687657 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0cad1-4da8-4f9d-911e-7d7ec9ae5be6 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:09.688434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0cad1-4da8-4f9d-911e-7d7ec9ae5be6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:09.690917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0cad1-4da8-4f9d-911e-7d7ec9ae5be6 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:09.718601 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1261/2495] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:12:09 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:09.722914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f6f56-89f4-4e9f-a8bb-1abfddc83e70 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:09.735030 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f6f56-89f4-4e9f-a8bb-1abfddc83e70 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:09.740369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be813de3-985a-4dd9-bcc5-1b44973229b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:09.741068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be813de3-985a-4dd9-bcc5-1b44973229b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:09.741356 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be813de3-985a-4dd9-bcc5-1b44973229b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:09.741636 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be813de3-985a-4dd9-bcc5-1b44973229b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:09.742357 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be813de3-985a-4dd9-bcc5-1b44973229b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:09.743956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be813de3-985a-4dd9-bcc5-1b44973229b3 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:09.755728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1235/2496] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:09.759743 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0cad1-4da8-4f9d-911e-7d7ec9ae5be6 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:09.770808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0cad1-4da8-4f9d-911e-7d7ec9ae5be6 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:09.781115 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a8067-88a3-4027-a597-52ac9ef5e603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:09.783097 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a8067-88a3-4027-a597-52ac9ef5e603 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:09.783733 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a8067-88a3-4027-a597-52ac9ef5e603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:09.784428 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a8067-88a3-4027-a597-52ac9ef5e603 admin admin] PATH_INFO: `/v3/projects/06e1747f1c4843c9979a69b634270162/users/c59694ea7ad640578366999daadc3537/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:09.804737 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a8067-88a3-4027-a597-52ac9ef5e603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e1747f1c4843c9979a69b634270162, user_id=c59694ea7ad640578366999daadc3537, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:09.809818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a8067-88a3-4027-a597-52ac9ef5e603 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:09.829496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1262/2497] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:12:09 2022] PUT /identity/v3/projects/06e1747f1c4843c9979a69b634270162/users/c59694ea7ad640578366999daadc3537/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:09.833835 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be813de3-985a-4dd9-bcc5-1b44973229b3 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:09.845575 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be813de3-985a-4dd9-bcc5-1b44973229b3 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:09.922130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5762e-d8df-4f47-8e21-01aa4d243e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:09.922835 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5762e-d8df-4f47-8e21-01aa4d243e50 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:09.923116 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5762e-d8df-4f47-8e21-01aa4d243e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:09.923390 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5762e-d8df-4f47-8e21-01aa4d243e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:09.924115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5762e-d8df-4f47-8e21-01aa4d243e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:09.925853 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5762e-d8df-4f47-8e21-01aa4d243e50 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:09.936875 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1236/2498] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:09.941196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a8067-88a3-4027-a597-52ac9ef5e603 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:09.953413 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a8067-88a3-4027-a597-52ac9ef5e603 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:09.958554 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2279fe3-e622-4bfe-b3c2-30b705108d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:09.959452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2279fe3-e622-4bfe-b3c2-30b705108d52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:09.959737 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2279fe3-e622-4bfe-b3c2-30b705108d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:09.960016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2279fe3-e622-4bfe-b3c2-30b705108d52 admin admin] PATH_INFO: `/v3/projects/06e1747f1c4843c9979a69b634270162/users/c59694ea7ad640578366999daadc3537/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:09.962060 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2279fe3-e622-4bfe-b3c2-30b705108d52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e1747f1c4843c9979a69b634270162, user_id=c59694ea7ad640578366999daadc3537, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:09.964750 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2279fe3-e622-4bfe-b3c2-30b705108d52 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:09.968587 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:09.977984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1263/2499] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:12:09 2022] PUT /identity/v3/projects/06e1747f1c4843c9979a69b634270162/users/c59694ea7ad640578366999daadc3537/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:09.989506 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6935a-35e1-4147-8d4c-9c3bf273183e None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:09.989796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6935a-35e1-4147-8d4c-9c3bf273183e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:09.990073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6935a-35e1-4147-8d4c-9c3bf273183e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:10.011264 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:10.026333 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-5aa6935a-35e1-4147-8d4c-9c3bf273183e None None] MFA Rules not processed for user `c59694ea7ad640578366999daadc3537`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:10.028195 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-5aa6935a-35e1-4147-8d4c-9c3bf273183e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:10.114228 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1237/2500] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:10.128277 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2279fe3-e622-4bfe-b3c2-30b705108d52 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:10.130127 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c2279fe3-e622-4bfe-b3c2-30b705108d52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:10.196872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2279fe3-e622-4bfe-b3c2-30b705108d52 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:10.201444 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed7230-85ea-4fdf-afe9-7306585c00b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:10.202272 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed7230-85ea-4fdf-afe9-7306585c00b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:10.202660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed7230-85ea-4fdf-afe9-7306585c00b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:10.203039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed7230-85ea-4fdf-afe9-7306585c00b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:10.216113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed7230-85ea-4fdf-afe9-7306585c00b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:10.217958 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed7230-85ea-4fdf-afe9-7306585c00b3 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:10.230794 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1264/2501] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:10.313650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6935a-35e1-4147-8d4c-9c3bf273183e None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:10.424596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6935a-35e1-4147-8d4c-9c3bf273183e None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:10.430599 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5811073-34f7-4f95-a02a-43e6756d9003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:10.431565 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5811073-34f7-4f95-a02a-43e6756d9003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:10.431925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5811073-34f7-4f95-a02a-43e6756d9003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:10.432281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5811073-34f7-4f95-a02a-43e6756d9003 admin admin] PATH_INFO: `/v3/users/47794b1db02c4f6882c7019608d8e50b` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:10.433628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5811073-34f7-4f95-a02a-43e6756d9003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47794b1db02c4f6882c7019608d8e50b)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:10.435730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5811073-34f7-4f95-a02a-43e6756d9003 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:10.506068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a5811073-34f7-4f95-a02a-43e6756d9003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47794b1db02c4f6882c7019608d8e50b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:10.514198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a5811073-34f7-4f95-a02a-43e6756d9003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47794b1db02c4f6882c7019608d8e50b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:10.523864 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a5811073-34f7-4f95-a02a-43e6756d9003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47794b1db02c4f6882c7019608d8e50b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:10.524738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a5811073-34f7-4f95-a02a-43e6756d9003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47794b1db02c4f6882c7019608d8e50b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:10.525498 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a5811073-34f7-4f95-a02a-43e6756d9003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47794b1db02c4f6882c7019608d8e50b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:10.547744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1238/2502] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:12:10 2022] DELETE /identity/v3/users/47794b1db02c4f6882c7019608d8e50b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:10.552404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d682214-f9a0-410b-8f6f-83300ed07027 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:10.552704 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d682214-f9a0-410b-8f6f-83300ed07027 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:10.553015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d682214-f9a0-410b-8f6f-83300ed07027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:10.571993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-2d682214-f9a0-410b-8f6f-83300ed07027 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:10.573304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2d682214-f9a0-410b-8f6f-83300ed07027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:10.633871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1265/2503] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:10.643548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5811073-34f7-4f95-a02a-43e6756d9003 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:10.645042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a5811073-34f7-4f95-a02a-43e6756d9003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:10.664126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:10.709632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5811073-34f7-4f95-a02a-43e6756d9003 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:10.714024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de4d7e-fc41-4853-8a88-5532b54ebeea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:10.714713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de4d7e-fc41-4853-8a88-5532b54ebeea service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:10.715127 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de4d7e-fc41-4853-8a88-5532b54ebeea service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:10.715420 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de4d7e-fc41-4853-8a88-5532b54ebeea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:10.727542 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de4d7e-fc41-4853-8a88-5532b54ebeea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:10.729231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de4d7e-fc41-4853-8a88-5532b54ebeea service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:10.749460 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1239/2504] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:10.878584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d682214-f9a0-410b-8f6f-83300ed07027 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:10.890144 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2d682214-f9a0-410b-8f6f-83300ed07027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:10.947221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de4d7e-fc41-4853-8a88-5532b54ebeea service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:10.948619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e3de4d7e-fc41-4853-8a88-5532b54ebeea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:10.973570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de4d7e-fc41-4853-8a88-5532b54ebeea service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:10.981459 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d947a6-a18d-46f7-ae61-a2f38fdc9e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:10.983076 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d947a6-a18d-46f7-ae61-a2f38fdc9e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:10.983361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d947a6-a18d-46f7-ae61-a2f38fdc9e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:10.983660 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d947a6-a18d-46f7-ae61-a2f38fdc9e5d admin admin] PATH_INFO: `/v3/users/85dea53a4a034f5981b638bf92f2cf3b` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:10.985529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d947a6-a18d-46f7-ae61-a2f38fdc9e5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=85dea53a4a034f5981b638bf92f2cf3b)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:10.987495 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d947a6-a18d-46f7-ae61-a2f38fdc9e5d admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:11.011860 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d682214-f9a0-410b-8f6f-83300ed07027 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:11.018861 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51facdb-b7ac-4f8c-818d-a7f3149380d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f1ef52f8eb42a59a50a69873b9ffaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:11.019788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51facdb-b7ac-4f8c-818d-a7f3149380d1 service placement] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:11.020073 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51facdb-b7ac-4f8c-818d-a7f3149380d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:11.020354 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51facdb-b7ac-4f8c-818d-a7f3149380d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:11.022303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a51facdb-b7ac-4f8c-818d-a7f3149380d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:11.036523 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51facdb-b7ac-4f8c-818d-a7f3149380d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:11.039228 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51facdb-b7ac-4f8c-818d-a7f3149380d1 service placement] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:11.063891 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1266/2505] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:12:11.077425 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c2d947a6-a18d-46f7-ae61-a2f38fdc9e5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85dea53a4a034f5981b638bf92f2cf3b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.083491 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51facdb-b7ac-4f8c-818d-a7f3149380d1 service placement] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:11.085648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a51facdb-b7ac-4f8c-818d-a7f3149380d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:11.091817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c2d947a6-a18d-46f7-ae61-a2f38fdc9e5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85dea53a4a034f5981b638bf92f2cf3b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.103320 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c2d947a6-a18d-46f7-ae61-a2f38fdc9e5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85dea53a4a034f5981b638bf92f2cf3b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.104344 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c2d947a6-a18d-46f7-ae61-a2f38fdc9e5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85dea53a4a034f5981b638bf92f2cf3b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.105515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-c2d947a6-a18d-46f7-ae61-a2f38fdc9e5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85dea53a4a034f5981b638bf92f2cf3b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.109790 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51facdb-b7ac-4f8c-818d-a7f3149380d1 service placement] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:11.116858 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:11.127964 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:11.128263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:11.128545 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e admin admin] PATH_INFO: `/v3/projects/61da8c681703405a8c7dbbc5a6bc671e` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:11.129986 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1240/2506] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:12:10 2022] DELETE /identity/v3/users/85dea53a4a034f5981b638bf92f2cf3b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:12:11.130702 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e admin admin] RBAC: Authorizing `identity:delete_project(project_id=61da8c681703405a8c7dbbc5a6bc671e)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:11.141022 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:11.168115 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:11.200309 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61da8c681703405a8c7dbbc5a6bc671e', 'request_id': 'req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.206416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61da8c681703405a8c7dbbc5a6bc671e', 'request_id': 'req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.214437 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61da8c681703405a8c7dbbc5a6bc671e', 'request_id': 'req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.227921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e admin admin] The token cache is being invalidate because project 61da8c681703405a8c7dbbc5a6bc671e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:12:11.228240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.228946 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.230559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1267/2507] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:12:11 2022] DELETE /identity/v3/projects/61da8c681703405a8c7dbbc5a6bc671e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:12:11.235652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d947a6-a18d-46f7-ae61-a2f38fdc9e5d admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:11.236992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c2d947a6-a18d-46f7-ae61-a2f38fdc9e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:11.313727 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d947a6-a18d-46f7-ae61-a2f38fdc9e5d admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:11.342681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:11.344132 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:11.354469 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:11.386650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893601a-d7d4-410f-be4d-ede64db96f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:11.387652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893601a-d7d4-410f-be4d-ede64db96f65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:11.387929 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893601a-d7d4-410f-be4d-ede64db96f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:11.388273 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893601a-d7d4-410f-be4d-ede64db96f65 admin admin] PATH_INFO: `/v3/users/c59694ea7ad640578366999daadc3537` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:11.390134 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893601a-d7d4-410f-be4d-ede64db96f65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c59694ea7ad640578366999daadc3537)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:11.392193 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893601a-d7d4-410f-be4d-ede64db96f65 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:11.418523 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dcd16-0451-4ec4-bc01-01bc5932c52e admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:11.424527 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d49b73-8b4d-4ac4-9759-75331ff1b019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:11.425454 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d49b73-8b4d-4ac4-9759-75331ff1b019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:11.425783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d49b73-8b4d-4ac4-9759-75331ff1b019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:11.426051 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d49b73-8b4d-4ac4-9759-75331ff1b019 admin admin] PATH_INFO: `/v3/projects/1e4d6faeb79c4834a4ca61193f0cc657` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:11.427318 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d49b73-8b4d-4ac4-9759-75331ff1b019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e4d6faeb79c4834a4ca61193f0cc657)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:11.429285 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d49b73-8b4d-4ac4-9759-75331ff1b019 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:11.446033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9893601a-d7d4-410f-be4d-ede64db96f65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c59694ea7ad640578366999daadc3537'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.455324 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9893601a-d7d4-410f-be4d-ede64db96f65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c59694ea7ad640578366999daadc3537'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.464397 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9893601a-d7d4-410f-be4d-ede64db96f65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c59694ea7ad640578366999daadc3537'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.465226 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9893601a-d7d4-410f-be4d-ede64db96f65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c59694ea7ad640578366999daadc3537'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.465893 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-9893601a-d7d4-410f-be4d-ede64db96f65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c59694ea7ad640578366999daadc3537'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.484065 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1241/2508] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:12:11 2022] DELETE /identity/v3/users/c59694ea7ad640578366999daadc3537 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:12:11.485582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-93d49b73-8b4d-4ac4-9759-75331ff1b019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e4d6faeb79c4834a4ca61193f0cc657', 'request_id': 'req-93d49b73-8b4d-4ac4-9759-75331ff1b019'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.491331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-93d49b73-8b4d-4ac4-9759-75331ff1b019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e4d6faeb79c4834a4ca61193f0cc657', 'request_id': 'req-93d49b73-8b4d-4ac4-9759-75331ff1b019'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.499811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-93d49b73-8b4d-4ac4-9759-75331ff1b019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e4d6faeb79c4834a4ca61193f0cc657', 'request_id': 'req-93d49b73-8b4d-4ac4-9759-75331ff1b019'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.511085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-93d49b73-8b4d-4ac4-9759-75331ff1b019 admin admin] The token cache is being invalidate because project 1e4d6faeb79c4834a4ca61193f0cc657 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:12:11.511375 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-93d49b73-8b4d-4ac4-9759-75331ff1b019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.512028 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-93d49b73-8b4d-4ac4-9759-75331ff1b019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.514632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1268/2509] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:12:11 2022] DELETE /identity/v3/projects/1e4d6faeb79c4834a4ca61193f0cc657 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:12:11.519790 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02544423-17d8-49d4-8d89-0e344e7022e9 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:11.520087 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02544423-17d8-49d4-8d89-0e344e7022e9 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:11.520370 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02544423-17d8-49d4-8d89-0e344e7022e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:11.539679 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-02544423-17d8-49d4-8d89-0e344e7022e9 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:11.540983 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-02544423-17d8-49d4-8d89-0e344e7022e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:11.603716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1242/2510] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:11.611593 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8eed5-f8d1-4898-8f90-3b5e4fbc8d8f None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:11.611904 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8eed5-f8d1-4898-8f90-3b5e4fbc8d8f None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:11.612186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8eed5-f8d1-4898-8f90-3b5e4fbc8d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:11.633001 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-13b8eed5-f8d1-4898-8f90-3b5e4fbc8d8f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:11.634247 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-13b8eed5-f8d1-4898-8f90-3b5e4fbc8d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:11.643047 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1269/2511] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:11.647713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02544423-17d8-49d4-8d89-0e344e7022e9 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:11.658543 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02544423-17d8-49d4-8d89-0e344e7022e9 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:11.663653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceef9c8-bba3-4896-9e35-fac86c30b280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:11.664313 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceef9c8-bba3-4896-9e35-fac86c30b280 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:11.664594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceef9c8-bba3-4896-9e35-fac86c30b280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:11.664898 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceef9c8-bba3-4896-9e35-fac86c30b280 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:11.665630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceef9c8-bba3-4896-9e35-fac86c30b280 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:12:11.665934 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceef9c8-bba3-4896-9e35-fac86c30b280 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:11.667526 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceef9c8-bba3-4896-9e35-fac86c30b280 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:11.679835 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1243/2512] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 22:12:11.684153 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8eed5-f8d1-4898-8f90-3b5e4fbc8d8f None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:11.695006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8eed5-f8d1-4898-8f90-3b5e4fbc8d8f None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:11.699766 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d904b1c-ec04-4785-9661-f446dbb56eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:11.700519 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d904b1c-ec04-4785-9661-f446dbb56eda admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:11.700851 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d904b1c-ec04-4785-9661-f446dbb56eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:11.701135 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d904b1c-ec04-4785-9661-f446dbb56eda admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:11.701833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d904b1c-ec04-4785-9661-f446dbb56eda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:11.703783 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d904b1c-ec04-4785-9661-f446dbb56eda admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:11.734435 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1270/2513] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:12:11 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:11.739062 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceef9c8-bba3-4896-9e35-fac86c30b280 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:11.754966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceef9c8-bba3-4896-9e35-fac86c30b280 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:11.813132 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d904b1c-ec04-4785-9661-f446dbb56eda admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:11.815209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-1d904b1c-ec04-4785-9661-f446dbb56eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:11.831100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:11.831272 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:11.855183 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86238ba-2ec7-4219-a819-755e3043dfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:11.856219 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86238ba-2ec7-4219-a819-755e3043dfbd admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:11.856572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86238ba-2ec7-4219-a819-755e3043dfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:11.857053 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86238ba-2ec7-4219-a819-755e3043dfbd admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:11.857943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86238ba-2ec7-4219-a819-755e3043dfbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:11.860054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86238ba-2ec7-4219-a819-755e3043dfbd admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:11.890607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1244/2514] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:12:11 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:12:11.894078 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86238ba-2ec7-4219-a819-755e3043dfbd admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:11.903134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d904b1c-ec04-4785-9661-f446dbb56eda admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:11.904924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86238ba-2ec7-4219-a819-755e3043dfbd admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:11.909645 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772eabee-8b2a-4534-a613-77d39a99cd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:11.909801 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ad867-5732-43ad-805f-8fa4f1ce9623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:11.910300 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ad867-5732-43ad-805f-8fa4f1ce9623 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:11.910507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772eabee-8b2a-4534-a613-77d39a99cd14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:11.910614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ad867-5732-43ad-805f-8fa4f1ce9623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:11.910801 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772eabee-8b2a-4534-a613-77d39a99cd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:11.910942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ad867-5732-43ad-805f-8fa4f1ce9623 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:11.911085 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772eabee-8b2a-4534-a613-77d39a99cd14 admin admin] PATH_INFO: `/v3/projects/06e1747f1c4843c9979a69b634270162` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:11.911541 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ad867-5732-43ad-805f-8fa4f1ce9623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:11.912293 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772eabee-8b2a-4534-a613-77d39a99cd14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e1747f1c4843c9979a69b634270162)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:11.913172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ad867-5732-43ad-805f-8fa4f1ce9623 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:11.914219 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772eabee-8b2a-4534-a613-77d39a99cd14 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:11.926254 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1245/2515] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:11.927297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ad867-5732-43ad-805f-8fa4f1ce9623 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:11.937441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ad867-5732-43ad-805f-8fa4f1ce9623 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:11.942118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763bc63-5600-4985-88e5-4a22367f9cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:11.942805 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763bc63-5600-4985-88e5-4a22367f9cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:11.943090 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763bc63-5600-4985-88e5-4a22367f9cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:11.943634 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763bc63-5600-4985-88e5-4a22367f9cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:11.944352 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763bc63-5600-4985-88e5-4a22367f9cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:11.946180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763bc63-5600-4985-88e5-4a22367f9cc8 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:11.958513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1246/2516] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:12:11.962413 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763bc63-5600-4985-88e5-4a22367f9cc8 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:11.968888 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-772eabee-8b2a-4534-a613-77d39a99cd14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e1747f1c4843c9979a69b634270162', 'request_id': 'req-772eabee-8b2a-4534-a613-77d39a99cd14'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.973410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763bc63-5600-4985-88e5-4a22367f9cc8 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:11.974683 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-772eabee-8b2a-4534-a613-77d39a99cd14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e1747f1c4843c9979a69b634270162', 'request_id': 'req-772eabee-8b2a-4534-a613-77d39a99cd14'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.984097 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-772eabee-8b2a-4534-a613-77d39a99cd14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e1747f1c4843c9979a69b634270162', 'request_id': 'req-772eabee-8b2a-4534-a613-77d39a99cd14'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.996253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-772eabee-8b2a-4534-a613-77d39a99cd14 admin admin] The token cache is being invalidate because project 06e1747f1c4843c9979a69b634270162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:12:11.996531 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-772eabee-8b2a-4534-a613-77d39a99cd14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.997246 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-772eabee-8b2a-4534-a613-77d39a99cd14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:11.999504 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1271/2517] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:12:11 2022] DELETE /identity/v3/projects/06e1747f1c4843c9979a69b634270162 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:12:12.004063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e990aa-c495-48d7-8b4b-48382e04fc8c None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:12.004347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e990aa-c495-48d7-8b4b-48382e04fc8c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:12.004621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e990aa-c495-48d7-8b4b-48382e04fc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:12.023476 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-b2e990aa-c495-48d7-8b4b-48382e04fc8c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:12.024736 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b2e990aa-c495-48d7-8b4b-48382e04fc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:12.036551 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72a3e9-4847-470d-b11c-9094e3498359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:12.037487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72a3e9-4847-470d-b11c-9094e3498359 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:12.037773 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72a3e9-4847-470d-b11c-9094e3498359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:12.038054 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72a3e9-4847-470d-b11c-9094e3498359 admin admin] PATH_INFO: `/v3/projects/8f382b789a354a558d9e84a899da2a50/users/4d89525cb4294b5ab7ea8d80eacbf837/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:12.057708 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a72a3e9-4847-470d-b11c-9094e3498359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f382b789a354a558d9e84a899da2a50, user_id=4d89525cb4294b5ab7ea8d80eacbf837, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:12.063317 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a72a3e9-4847-470d-b11c-9094e3498359 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:12.078112 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1247/2518] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:12:11 2022] PUT /identity/v3/projects/8f382b789a354a558d9e84a899da2a50/users/4d89525cb4294b5ab7ea8d80eacbf837/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:12:12.089132 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fd725-a66c-45f5-9984-9487f66d5ec6 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:12.089651 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1272/2519] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:12:12.089823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fd725-a66c-45f5-9984-9487f66d5ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:12.093325 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fd725-a66c-45f5-9984-9487f66d5ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:12.097656 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8836c6-725a-4983-afb8-578e25696c7b None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:12.097949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8836c6-725a-4983-afb8-578e25696c7b None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:12.098251 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8836c6-725a-4983-afb8-578e25696c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:12.117056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-2a8836c6-725a-4983-afb8-578e25696c7b None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:12.118324 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2a8836c6-725a-4983-afb8-578e25696c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:12.125913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-754fd725-a66c-45f5-9984-9487f66d5ec6 None None] MFA Rules not processed for user `4d89525cb4294b5ab7ea8d80eacbf837`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:12.127302 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-754fd725-a66c-45f5-9984-9487f66d5ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:12.166212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:12.192407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1273/2520] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:12:12.196583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8836c6-725a-4983-afb8-578e25696c7b None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:12.206760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8836c6-725a-4983-afb8-578e25696c7b None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:12.207417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1248/2521] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:12:12.214838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9d066-8dff-4159-ad06-eabf372df5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:12.215509 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9d066-8dff-4159-ad06-eabf372df5cb admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:12.215791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9d066-8dff-4159-ad06-eabf372df5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:12.216602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9d066-8dff-4159-ad06-eabf372df5cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:12.217479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9d066-8dff-4159-ad06-eabf372df5cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:12:12.217686 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9d066-8dff-4159-ad06-eabf372df5cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:12.219510 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9d066-8dff-4159-ad06-eabf372df5cb admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:12.220672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fd725-a66c-45f5-9984-9487f66d5ec6 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:12.222212 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-754fd725-a66c-45f5-9984-9487f66d5ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:12.232849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1274/2522] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:12:12.237235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9d066-8dff-4159-ad06-eabf372df5cb admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:12.249487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9d066-8dff-4159-ad06-eabf372df5cb admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:12.254071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:12.256060 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3dab3-35c0-4920-8a88-744692c226b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:12.256744 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3dab3-35c0-4920-8a88-744692c226b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:12.257041 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3dab3-35c0-4920-8a88-744692c226b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:12.257452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3dab3-35c0-4920-8a88-744692c226b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:12.258321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3dab3-35c0-4920-8a88-744692c226b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:12.260963 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3dab3-35c0-4920-8a88-744692c226b5 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:12.296070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1275/2523] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:12:12 2022] POST /identity/v3/projects => generated 401 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:12:12.300164 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3dab3-35c0-4920-8a88-744692c226b5 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:12.307655 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fd725-a66c-45f5-9984-9487f66d5ec6 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:12.368821 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3dab3-35c0-4920-8a88-744692c226b5 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:12.373930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19ca07-5a0f-46b4-b8a4-29fbf54b94d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:12.374636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19ca07-5a0f-46b4-b8a4-29fbf54b94d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:12.374924 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19ca07-5a0f-46b4-b8a4-29fbf54b94d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:12.375204 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19ca07-5a0f-46b4-b8a4-29fbf54b94d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:12.375904 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19ca07-5a0f-46b4-b8a4-29fbf54b94d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:12.376048 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df392aa-275f-45b7-8823-bcb18013a719 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:12.376653 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df392aa-275f-45b7-8823-bcb18013a719 service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:12.377086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df392aa-275f-45b7-8823-bcb18013a719 service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:12.377376 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df392aa-275f-45b7-8823-bcb18013a719 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:12.377866 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19ca07-5a0f-46b4-b8a4-29fbf54b94d8 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:12.402605 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1276/2524] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:12:12 2022] POST /identity/v3/users => generated 301 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:12:12.409355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19ca07-5a0f-46b4-b8a4-29fbf54b94d8 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:12.420178 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19ca07-5a0f-46b4-b8a4-29fbf54b94d8 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:12.426016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea40b4-e62c-47aa-976d-24fb3969cbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:12.426692 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea40b4-e62c-47aa-976d-24fb3969cbac admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:12.426982 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea40b4-e62c-47aa-976d-24fb3969cbac admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:12.427264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea40b4-e62c-47aa-976d-24fb3969cbac admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:12.427993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea40b4-e62c-47aa-976d-24fb3969cbac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:12.429645 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea40b4-e62c-47aa-976d-24fb3969cbac admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:12.440333 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1277/2525] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:12.444770 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea40b4-e62c-47aa-976d-24fb3969cbac admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:12.447225 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df392aa-275f-45b7-8823-bcb18013a719 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:12.448984 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df392aa-275f-45b7-8823-bcb18013a719 service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:12.455324 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea40b4-e62c-47aa-976d-24fb3969cbac admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:12.459340 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1249/2526] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:12:12.460170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6db723-c4d5-4163-8e8b-b9b516f1d0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:12.460864 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6db723-c4d5-4163-8e8b-b9b516f1d0e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:12.461160 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6db723-c4d5-4163-8e8b-b9b516f1d0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:12.461442 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6db723-c4d5-4163-8e8b-b9b516f1d0e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:12.462174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6db723-c4d5-4163-8e8b-b9b516f1d0e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:12.463795 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6db723-c4d5-4163-8e8b-b9b516f1d0e0 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:12.479801 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1278/2527] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:12:12.480832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df392aa-275f-45b7-8823-bcb18013a719 service nova] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:12.491777 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df392aa-275f-45b7-8823-bcb18013a719 service nova] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:12.496609 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab8ba0-b8f3-4383-a3e2-a54047d21cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:12.497597 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab8ba0-b8f3-4383-a3e2-a54047d21cea admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:12.497881 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab8ba0-b8f3-4383-a3e2-a54047d21cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:12.498262 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab8ba0-b8f3-4383-a3e2-a54047d21cea admin admin] PATH_INFO: `/v3/projects/b7997c17cfe44d5b937aa9f2f802f1e9/users/f77be1c81f034967b0f6a26f74e4a446/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:12.514250 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab8ba0-b8f3-4383-a3e2-a54047d21cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7997c17cfe44d5b937aa9f2f802f1e9, user_id=f77be1c81f034967b0f6a26f74e4a446, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:12.516968 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab8ba0-b8f3-4383-a3e2-a54047d21cea admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:12.532728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1250/2528] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:12:12 2022] PUT /identity/v3/projects/b7997c17cfe44d5b937aa9f2f802f1e9/users/f77be1c81f034967b0f6a26f74e4a446/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:12:12.543204 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afbd33-6318-4b6b-aa47-405c5e59fab3 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:12.543582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afbd33-6318-4b6b-aa47-405c5e59fab3 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:12.543951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afbd33-6318-4b6b-aa47-405c5e59fab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:12.571327 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-90afbd33-6318-4b6b-aa47-405c5e59fab3 None None] MFA Rules not processed for user `f77be1c81f034967b0f6a26f74e4a446`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:12.572583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-90afbd33-6318-4b6b-aa47-405c5e59fab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:12.643981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1279/2529] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:12:12.649020 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab8ba0-b8f3-4383-a3e2-a54047d21cea admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:12.690823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:12.715441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab8ba0-b8f3-4383-a3e2-a54047d21cea admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:12.721280 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223940f3-d900-4845-aa5a-906ca61a570d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:12.721954 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223940f3-d900-4845-aa5a-906ca61a570d admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:12.722246 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223940f3-d900-4845-aa5a-906ca61a570d admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:12.722587 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223940f3-d900-4845-aa5a-906ca61a570d admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:12.723290 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223940f3-d900-4845-aa5a-906ca61a570d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:12.725657 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223940f3-d900-4845-aa5a-906ca61a570d admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:12.755828 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1251/2530] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:12:12 2022] POST /identity/v3/projects => generated 399 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:12:12.760147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90afbd33-6318-4b6b-aa47-405c5e59fab3 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:12.782822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90afbd33-6318-4b6b-aa47-405c5e59fab3 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:12.787477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:12.864615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099df53-2319-4166-963f-d1c380dbadc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:12.865307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099df53-2319-4166-963f-d1c380dbadc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:12.865600 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099df53-2319-4166-963f-d1c380dbadc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:12.865885 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099df53-2319-4166-963f-d1c380dbadc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:12.866609 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099df53-2319-4166-963f-d1c380dbadc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:12.868665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099df53-2319-4166-963f-d1c380dbadc0 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:12.895220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1280/2531] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:12:12 2022] POST /identity/v3/users => generated 306 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:12.896411 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223940f3-d900-4845-aa5a-906ca61a570d admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:12.907520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223940f3-d900-4845-aa5a-906ca61a570d admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:12.912365 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd85797-74b5-426b-a495-9fbe580c689f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:12.913068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd85797-74b5-426b-a495-9fbe580c689f admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:12.913356 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd85797-74b5-426b-a495-9fbe580c689f admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:12.913636 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd85797-74b5-426b-a495-9fbe580c689f admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:12.914407 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd85797-74b5-426b-a495-9fbe580c689f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:12.916067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd85797-74b5-426b-a495-9fbe580c689f admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:12.927850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1252/2532] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:12.932989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099df53-2319-4166-963f-d1c380dbadc0 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:12.945586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099df53-2319-4166-963f-d1c380dbadc0 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:12.950680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2434fc64-05af-490c-bc2f-9479b30a9a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:12.951602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434fc64-05af-490c-bc2f-9479b30a9a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:12.951894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434fc64-05af-490c-bc2f-9479b30a9a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:12.952179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434fc64-05af-490c-bc2f-9479b30a9a27 admin admin] PATH_INFO: `/v3/projects/cad23508ac0041a5954208f46ae8cce7/users/ca89f6ea991f4c8a8e826e2946d165dc/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:12.968642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2434fc64-05af-490c-bc2f-9479b30a9a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cad23508ac0041a5954208f46ae8cce7, user_id=ca89f6ea991f4c8a8e826e2946d165dc, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:12.971432 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2434fc64-05af-490c-bc2f-9479b30a9a27 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:12.985916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1281/2533] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:12:12 2022] PUT /identity/v3/projects/cad23508ac0041a5954208f46ae8cce7/users/ca89f6ea991f4c8a8e826e2946d165dc/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:12:12.989957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd85797-74b5-426b-a495-9fbe580c689f admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:13.000365 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd85797-74b5-426b-a495-9fbe580c689f admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:13.063091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00570bad-7dbe-42e3-8b06-ed4ee764c9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:13.063798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00570bad-7dbe-42e3-8b06-ed4ee764c9b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:13.064086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00570bad-7dbe-42e3-8b06-ed4ee764c9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:13.064364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00570bad-7dbe-42e3-8b06-ed4ee764c9b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:13.065127 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00570bad-7dbe-42e3-8b06-ed4ee764c9b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:13.066855 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00570bad-7dbe-42e3-8b06-ed4ee764c9b2 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:13.083732 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1253/2534] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:13.088297 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2434fc64-05af-490c-bc2f-9479b30a9a27 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:13.103567 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2434fc64-05af-490c-bc2f-9479b30a9a27 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:13.114959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b46c7e-2250-4d51-ac3b-809719ec6fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:13.116648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b46c7e-2250-4d51-ac3b-809719ec6fb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:13.117260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b46c7e-2250-4d51-ac3b-809719ec6fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:13.117713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b46c7e-2250-4d51-ac3b-809719ec6fb5 admin admin] PATH_INFO: `/v3/projects/cad23508ac0041a5954208f46ae8cce7/users/ca89f6ea991f4c8a8e826e2946d165dc/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:13.124591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b46c7e-2250-4d51-ac3b-809719ec6fb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cad23508ac0041a5954208f46ae8cce7, user_id=ca89f6ea991f4c8a8e826e2946d165dc, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:13.124998 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b46c7e-2250-4d51-ac3b-809719ec6fb5 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:13.140744 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1282/2535] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:12:13 2022] PUT /identity/v3/projects/cad23508ac0041a5954208f46ae8cce7/users/ca89f6ea991f4c8a8e826e2946d165dc/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:13.208896 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccef07-9e89-43ea-8a17-25aee23e859f None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:13.209494 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccef07-9e89-43ea-8a17-25aee23e859f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:13.209787 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccef07-9e89-43ea-8a17-25aee23e859f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:13.240607 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-32ccef07-9e89-43ea-8a17-25aee23e859f None None] MFA Rules not processed for user `ca89f6ea991f4c8a8e826e2946d165dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:13.241903 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-32ccef07-9e89-43ea-8a17-25aee23e859f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:13.316036 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1254/2536] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:13 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:13.328263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b46c7e-2250-4d51-ac3b-809719ec6fb5 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:13.330096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-79b46c7e-2250-4d51-ac3b-809719ec6fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:13.387342 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b46c7e-2250-4d51-ac3b-809719ec6fb5 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:13.391862 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a78aac-3abb-433e-90d4-8e4aa53bc0a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:13.392506 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a78aac-3abb-433e-90d4-8e4aa53bc0a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:13.392819 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a78aac-3abb-433e-90d4-8e4aa53bc0a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:13.393104 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a78aac-3abb-433e-90d4-8e4aa53bc0a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:13.403386 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a78aac-3abb-433e-90d4-8e4aa53bc0a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:13.405630 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a78aac-3abb-433e-90d4-8e4aa53bc0a3 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:13.415305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1283/2537] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:13 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:13.488307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccef07-9e89-43ea-8a17-25aee23e859f None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:13.498596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccef07-9e89-43ea-8a17-25aee23e859f None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:13.503569 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032f22a-347c-449b-b511-fa5685266971 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:13.504646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032f22a-347c-449b-b511-fa5685266971 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:13.505185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032f22a-347c-449b-b511-fa5685266971 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:13.505628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032f22a-347c-449b-b511-fa5685266971 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:13.536836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:13.567217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032f22a-347c-449b-b511-fa5685266971 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:13.568971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032f22a-347c-449b-b511-fa5685266971 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:13.578394 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1255/2538] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:13 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:17.627084 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a78aac-3abb-433e-90d4-8e4aa53bc0a3 service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:17.719201 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a78aac-3abb-433e-90d4-8e4aa53bc0a3 service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:17.725200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed63246-eeea-4145-87de-4388b439874d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:17.726273 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed63246-eeea-4145-87de-4388b439874d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:17.726665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed63246-eeea-4145-87de-4388b439874d admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:17.727222 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed63246-eeea-4145-87de-4388b439874d admin admin] PATH_INFO: `/v3/users/f77be1c81f034967b0f6a26f74e4a446` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:17.730157 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed63246-eeea-4145-87de-4388b439874d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f77be1c81f034967b0f6a26f74e4a446)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:17.734125 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed63246-eeea-4145-87de-4388b439874d admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:17.737374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:17.805503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-aed63246-eeea-4145-87de-4388b439874d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f77be1c81f034967b0f6a26f74e4a446'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:17.830363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-aed63246-eeea-4145-87de-4388b439874d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f77be1c81f034967b0f6a26f74e4a446'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:17.841526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-aed63246-eeea-4145-87de-4388b439874d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f77be1c81f034967b0f6a26f74e4a446'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:17.850900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-aed63246-eeea-4145-87de-4388b439874d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f77be1c81f034967b0f6a26f74e4a446'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:17.851689 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-aed63246-eeea-4145-87de-4388b439874d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f77be1c81f034967b0f6a26f74e4a446'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:17.869391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1284/2539] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:12:17 2022] DELETE /identity/v3/users/f77be1c81f034967b0f6a26f74e4a446 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:17.871338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f89b2e-85be-4e56-a1d9-1075600f9c55 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:17.873247 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f89b2e-85be-4e56-a1d9-1075600f9c55 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:17.873537 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f89b2e-85be-4e56-a1d9-1075600f9c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:18.026877 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-54f89b2e-85be-4e56-a1d9-1075600f9c55 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:18.028168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-54f89b2e-85be-4e56-a1d9-1075600f9c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:18.102783 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1256/2540] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:18.113961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed63246-eeea-4145-87de-4388b439874d admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:18.115740 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-aed63246-eeea-4145-87de-4388b439874d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:18.207314 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed63246-eeea-4145-87de-4388b439874d admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:18.211826 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20eb74-252c-4353-89c1-544f10a6d09c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:18.212495 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20eb74-252c-4353-89c1-544f10a6d09c service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:18.212830 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20eb74-252c-4353-89c1-544f10a6d09c service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:18.213119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20eb74-252c-4353-89c1-544f10a6d09c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:18.223831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20eb74-252c-4353-89c1-544f10a6d09c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:18.225432 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20eb74-252c-4353-89c1-544f10a6d09c service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:18.236935 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1285/2541] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:18.551149 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f89b2e-85be-4e56-a1d9-1075600f9c55 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:18.552561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-54f89b2e-85be-4e56-a1d9-1075600f9c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:18.566517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f89b2e-85be-4e56-a1d9-1075600f9c55 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:18.571684 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:18.572690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:18.573014 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:18.573295 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71 admin admin] PATH_INFO: `/v3/projects/b7997c17cfe44d5b937aa9f2f802f1e9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:18.576674 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7997c17cfe44d5b937aa9f2f802f1e9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:18.580651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:18.640404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7997c17cfe44d5b937aa9f2f802f1e9', 'request_id': 'req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:18.647657 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7997c17cfe44d5b937aa9f2f802f1e9', 'request_id': 'req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:18.657919 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7997c17cfe44d5b937aa9f2f802f1e9', 'request_id': 'req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:18.670127 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71 admin admin] The token cache is being invalidate because project b7997c17cfe44d5b937aa9f2f802f1e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:12:18.670545 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:18.671232 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:18.672810 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1257/2542] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:12:18 2022] DELETE /identity/v3/projects/b7997c17cfe44d5b937aa9f2f802f1e9 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:18.677018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20eb74-252c-4353-89c1-544f10a6d09c service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:18.687556 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20eb74-252c-4353-89c1-544f10a6d09c service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:18.753137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739bb062-3ade-447c-becd-40a859a6eaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:18.754331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bb062-3ade-447c-becd-40a859a6eaa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:18.754628 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bb062-3ade-447c-becd-40a859a6eaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:18.754959 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bb062-3ade-447c-becd-40a859a6eaa1 admin admin] PATH_INFO: `/v3/users/ca89f6ea991f4c8a8e826e2946d165dc` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:18.756384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739bb062-3ade-447c-becd-40a859a6eaa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca89f6ea991f4c8a8e826e2946d165dc)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:18.758524 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739bb062-3ade-447c-becd-40a859a6eaa1 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:18.790654 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:18.816411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-739bb062-3ade-447c-becd-40a859a6eaa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca89f6ea991f4c8a8e826e2946d165dc'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:18.829833 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-739bb062-3ade-447c-becd-40a859a6eaa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca89f6ea991f4c8a8e826e2946d165dc'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:18.837223 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-739bb062-3ade-447c-becd-40a859a6eaa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca89f6ea991f4c8a8e826e2946d165dc'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:18.846191 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-739bb062-3ade-447c-becd-40a859a6eaa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca89f6ea991f4c8a8e826e2946d165dc'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:18.846848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-739bb062-3ade-447c-becd-40a859a6eaa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca89f6ea991f4c8a8e826e2946d165dc'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:18.853964 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1286/2543] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:12:18 2022] DELETE /identity/v3/users/ca89f6ea991f4c8a8e826e2946d165dc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:19.159195 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:19.161055 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:19.177624 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8b6c5-72e2-4c15-8f6a-c4546dc01b71 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:19.246815 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:19.258314 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4beac-19d5-410c-b28f-44be6408be36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:19.259193 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4beac-19d5-410c-b28f-44be6408be36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:19.259477 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4beac-19d5-410c-b28f-44be6408be36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:19.259756 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4beac-19d5-410c-b28f-44be6408be36 admin admin] PATH_INFO: `/v3/projects/cad23508ac0041a5954208f46ae8cce7` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:19.261016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4beac-19d5-410c-b28f-44be6408be36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cad23508ac0041a5954208f46ae8cce7)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:19.263589 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4beac-19d5-410c-b28f-44be6408be36 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:19.313304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-cdf4beac-19d5-410c-b28f-44be6408be36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cad23508ac0041a5954208f46ae8cce7', 'request_id': 'req-cdf4beac-19d5-410c-b28f-44be6408be36'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:19.319676 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-cdf4beac-19d5-410c-b28f-44be6408be36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cad23508ac0041a5954208f46ae8cce7', 'request_id': 'req-cdf4beac-19d5-410c-b28f-44be6408be36'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:19.328197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-cdf4beac-19d5-410c-b28f-44be6408be36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cad23508ac0041a5954208f46ae8cce7', 'request_id': 'req-cdf4beac-19d5-410c-b28f-44be6408be36'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:19.340509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-cdf4beac-19d5-410c-b28f-44be6408be36 admin admin] The token cache is being invalidate because project cad23508ac0041a5954208f46ae8cce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:12:19.340837 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-cdf4beac-19d5-410c-b28f-44be6408be36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:19.341572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-cdf4beac-19d5-410c-b28f-44be6408be36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:19.343126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1258/2544] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:12:19 2022] DELETE /identity/v3/projects/cad23508ac0041a5954208f46ae8cce7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:19.348021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771c375-d2bc-46a3-a9d7-d9c12457d395 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:19.348414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771c375-d2bc-46a3-a9d7-d9c12457d395 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:19.349318 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771c375-d2bc-46a3-a9d7-d9c12457d395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:19.367910 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-5771c375-d2bc-46a3-a9d7-d9c12457d395 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:19.369169 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5771c375-d2bc-46a3-a9d7-d9c12457d395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:19.440743 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1287/2545] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:19.446820 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5570e-e2e7-475c-a4d6-a767e8e0ab1a None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:19.447209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5570e-e2e7-475c-a4d6-a767e8e0ab1a None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:19.447588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5570e-e2e7-475c-a4d6-a767e8e0ab1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:19.465848 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-d4e5570e-e2e7-475c-a4d6-a767e8e0ab1a None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:19.467059 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d4e5570e-e2e7-475c-a4d6-a767e8e0ab1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:19.475028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1259/2546] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:19.479201 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771c375-d2bc-46a3-a9d7-d9c12457d395 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:19.489997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771c375-d2bc-46a3-a9d7-d9c12457d395 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:19.494960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e440bb-815b-4745-a754-83aead908bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:19.495621 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e440bb-815b-4745-a754-83aead908bfc admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:19.495911 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e440bb-815b-4745-a754-83aead908bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:19.496194 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e440bb-815b-4745-a754-83aead908bfc admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:19.497000 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e440bb-815b-4745-a754-83aead908bfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:12:19.498134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e440bb-815b-4745-a754-83aead908bfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:19.499829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e440bb-815b-4745-a754-83aead908bfc admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:19.515201 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1288/2547] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:12:19.516823 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5570e-e2e7-475c-a4d6-a767e8e0ab1a None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:19.527156 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5570e-e2e7-475c-a4d6-a767e8e0ab1a None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:19.532347 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55edb8a-4383-4136-a182-ce4f1e119081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:19.533052 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55edb8a-4383-4136-a182-ce4f1e119081 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:19.533360 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55edb8a-4383-4136-a182-ce4f1e119081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:19.533634 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55edb8a-4383-4136-a182-ce4f1e119081 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:19.534321 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55edb8a-4383-4136-a182-ce4f1e119081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:19.536251 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55edb8a-4383-4136-a182-ce4f1e119081 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:19.563130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1260/2548] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:12:19 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:19.567409 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e440bb-815b-4745-a754-83aead908bfc admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:19.578647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e440bb-815b-4745-a754-83aead908bfc admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:19.650092 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16805d07-a49c-420f-ba67-4ed452d8bae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:19.650773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16805d07-a49c-420f-ba67-4ed452d8bae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:19.651057 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16805d07-a49c-420f-ba67-4ed452d8bae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:19.651331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16805d07-a49c-420f-ba67-4ed452d8bae0 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:19.652064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16805d07-a49c-420f-ba67-4ed452d8bae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:19.654105 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16805d07-a49c-420f-ba67-4ed452d8bae0 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:19.675878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1289/2549] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:12:19 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:19.679917 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55edb8a-4383-4136-a182-ce4f1e119081 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:19.694143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55edb8a-4383-4136-a182-ce4f1e119081 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:19.699466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b72f1-c43b-4495-9740-61f50fc54482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:19.700902 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b72f1-c43b-4495-9740-61f50fc54482 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:19.701423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b72f1-c43b-4495-9740-61f50fc54482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:19.701791 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b72f1-c43b-4495-9740-61f50fc54482 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:19.702518 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b72f1-c43b-4495-9740-61f50fc54482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:19.704119 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b72f1-c43b-4495-9740-61f50fc54482 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:19.715479 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1261/2550] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:19.719901 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16805d07-a49c-420f-ba67-4ed452d8bae0 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:19.731503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16805d07-a49c-420f-ba67-4ed452d8bae0 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:19.739080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc9528-a0aa-41d7-8f27-b35e742d4eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:19.742849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc9528-a0aa-41d7-8f27-b35e742d4eee admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:19.742849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc9528-a0aa-41d7-8f27-b35e742d4eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:19.742849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc9528-a0aa-41d7-8f27-b35e742d4eee admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:19.742849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc9528-a0aa-41d7-8f27-b35e742d4eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:19.743623 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc9528-a0aa-41d7-8f27-b35e742d4eee admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:19.760767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1290/2551] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:19.764278 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b72f1-c43b-4495-9740-61f50fc54482 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:19.789829 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b72f1-c43b-4495-9740-61f50fc54482 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:19.799404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec667b72-aeab-47f2-ba20-65c221b29059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:19.801532 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec667b72-aeab-47f2-ba20-65c221b29059 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:19.802265 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec667b72-aeab-47f2-ba20-65c221b29059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:19.802988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec667b72-aeab-47f2-ba20-65c221b29059 admin admin] PATH_INFO: `/v3/projects/6a96f4f546604069a96d7b4fa8b64900/users/6fc8ec2fda70494c8a705f17f40deb40/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:19.828665 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec667b72-aeab-47f2-ba20-65c221b29059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a96f4f546604069a96d7b4fa8b64900, user_id=6fc8ec2fda70494c8a705f17f40deb40, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:19.830513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec667b72-aeab-47f2-ba20-65c221b29059 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:19.845968 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1262/2552] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:12:19 2022] PUT /identity/v3/projects/6a96f4f546604069a96d7b4fa8b64900/users/6fc8ec2fda70494c8a705f17f40deb40/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:12:19.858936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5ceff-b536-4ee0-bba1-66576b862f24 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:19.859192 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5ceff-b536-4ee0-bba1-66576b862f24 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:19.859470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5ceff-b536-4ee0-bba1-66576b862f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:19.873265 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec667b72-aeab-47f2-ba20-65c221b29059 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:19.873265 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ec667b72-aeab-47f2-ba20-65c221b29059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:19.879725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:19.894072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-0cb5ceff-b536-4ee0-bba1-66576b862f24 None None] MFA Rules not processed for user `6fc8ec2fda70494c8a705f17f40deb40`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:19.896160 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0cb5ceff-b536-4ee0-bba1-66576b862f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:19.949570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec667b72-aeab-47f2-ba20-65c221b29059 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:19.954535 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725cedd-897d-495c-b620-eaa149635735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:19.955419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725cedd-897d-495c-b620-eaa149635735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:19.955705 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725cedd-897d-495c-b620-eaa149635735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:19.956000 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725cedd-897d-495c-b620-eaa149635735 admin admin] PATH_INFO: `/v3/users/4d89525cb4294b5ab7ea8d80eacbf837` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:19.957445 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725cedd-897d-495c-b620-eaa149635735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d89525cb4294b5ab7ea8d80eacbf837)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:19.959725 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725cedd-897d-495c-b620-eaa149635735 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:19.978291 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1291/2553] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:19.987956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:19.989418 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5ceff-b536-4ee0-bba1-66576b862f24 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:19.991120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0cb5ceff-b536-4ee0-bba1-66576b862f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:20.015431 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0725cedd-897d-495c-b620-eaa149635735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d89525cb4294b5ab7ea8d80eacbf837'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:20.022455 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0725cedd-897d-495c-b620-eaa149635735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d89525cb4294b5ab7ea8d80eacbf837'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:20.030428 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0725cedd-897d-495c-b620-eaa149635735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d89525cb4294b5ab7ea8d80eacbf837'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:20.031263 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0725cedd-897d-495c-b620-eaa149635735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d89525cb4294b5ab7ea8d80eacbf837'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:20.031966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0725cedd-897d-495c-b620-eaa149635735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d89525cb4294b5ab7ea8d80eacbf837'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:20.051062 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1263/2554] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:12:19 2022] DELETE /identity/v3/users/4d89525cb4294b5ab7ea8d80eacbf837 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:12:20.053846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5ceff-b536-4ee0-bba1-66576b862f24 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:20.055355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571519f-6f64-4916-9126-c2b58e923be0 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:20.055722 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571519f-6f64-4916-9126-c2b58e923be0 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:20.056077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571519f-6f64-4916-9126-c2b58e923be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:20.075421 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-4571519f-6f64-4916-9126-c2b58e923be0 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:20.076753 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4571519f-6f64-4916-9126-c2b58e923be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:20.102577 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d4db8-d54f-454a-b43d-8a40c19734f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:20.103231 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d4db8-d54f-454a-b43d-8a40c19734f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:20.103521 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d4db8-d54f-454a-b43d-8a40c19734f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:20.103795 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d4db8-d54f-454a-b43d-8a40c19734f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:20.105589 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c24d4db8-d54f-454a-b43d-8a40c19734f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:20.144579 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1264/2555] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:12:20.160204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571519f-6f64-4916-9126-c2b58e923be0 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:20.161709 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4571519f-6f64-4916-9126-c2b58e923be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:20.178658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d4db8-d54f-454a-b43d-8a40c19734f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:20.180112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d4db8-d54f-454a-b43d-8a40c19734f5 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:20.189986 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1292/2556] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:12:20.221122 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571519f-6f64-4916-9126-c2b58e923be0 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:20.225069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae0604-54c8-4622-91e7-847a30e4d0a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:20.225719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae0604-54c8-4622-91e7-847a30e4d0a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:20.226126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae0604-54c8-4622-91e7-847a30e4d0a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:20.226412 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae0604-54c8-4622-91e7-847a30e4d0a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:20.236878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae0604-54c8-4622-91e7-847a30e4d0a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:20.238240 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae0604-54c8-4622-91e7-847a30e4d0a1 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:20.249071 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1265/2557] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:12:20.554763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d4db8-d54f-454a-b43d-8a40c19734f5 service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:20.556332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c24d4db8-d54f-454a-b43d-8a40c19734f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:20.585221 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d4db8-d54f-454a-b43d-8a40c19734f5 service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:20.591889 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f43e18-c254-4881-9040-7a3cf559a5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:20.593000 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f43e18-c254-4881-9040-7a3cf559a5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:20.593353 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f43e18-c254-4881-9040-7a3cf559a5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:20.593702 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f43e18-c254-4881-9040-7a3cf559a5e1 admin admin] PATH_INFO: `/v3/projects/8f382b789a354a558d9e84a899da2a50` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:20.595172 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f43e18-c254-4881-9040-7a3cf559a5e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f382b789a354a558d9e84a899da2a50)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:20.597262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f43e18-c254-4881-9040-7a3cf559a5e1 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:20.600098 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=100 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:20.649136 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f3f43e18-c254-4881-9040-7a3cf559a5e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f382b789a354a558d9e84a899da2a50', 'request_id': 'req-f3f43e18-c254-4881-9040-7a3cf559a5e1'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:20.654768 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f3f43e18-c254-4881-9040-7a3cf559a5e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f382b789a354a558d9e84a899da2a50', 'request_id': 'req-f3f43e18-c254-4881-9040-7a3cf559a5e1'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:20.662143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f3f43e18-c254-4881-9040-7a3cf559a5e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f382b789a354a558d9e84a899da2a50', 'request_id': 'req-f3f43e18-c254-4881-9040-7a3cf559a5e1'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:20.673354 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f3f43e18-c254-4881-9040-7a3cf559a5e1 admin admin] The token cache is being invalidate because project 8f382b789a354a558d9e84a899da2a50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:12:20.673630 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f3f43e18-c254-4881-9040-7a3cf559a5e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:20.674277 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f3f43e18-c254-4881-9040-7a3cf559a5e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:20.675835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1293/2558] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:12:20 2022] DELETE /identity/v3/projects/8f382b789a354a558d9e84a899da2a50 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:20.680870 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db4361-fdb2-4b9c-af42-03791aa04d89 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:20.681148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db4361-fdb2-4b9c-af42-03791aa04d89 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:20.681429 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db4361-fdb2-4b9c-af42-03791aa04d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:20.700119 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-a9db4361-fdb2-4b9c-af42-03791aa04d89 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:20.701933 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a9db4361-fdb2-4b9c-af42-03791aa04d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:20.741248 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:20.764344 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1266/2559] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:20.772306 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613c451-c480-4dc9-9c4a-1c7c84f7b034 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:20.772879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613c451-c480-4dc9-9c4a-1c7c84f7b034 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:20.773049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613c451-c480-4dc9-9c4a-1c7c84f7b034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:20.793437 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-b613c451-c480-4dc9-9c4a-1c7c84f7b034 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:20.795183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b613c451-c480-4dc9-9c4a-1c7c84f7b034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:20.803770 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1294/2560] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:12:20.808736 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db4361-fdb2-4b9c-af42-03791aa04d89 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:20.819178 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db4361-fdb2-4b9c-af42-03791aa04d89 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:20.824846 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d08214-28fe-4683-b911-456fbf681879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:20.826194 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d08214-28fe-4683-b911-456fbf681879 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:20.826481 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d08214-28fe-4683-b911-456fbf681879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:20.826766 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d08214-28fe-4683-b911-456fbf681879 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:20.827503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d08214-28fe-4683-b911-456fbf681879 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:12:20.827817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d08214-28fe-4683-b911-456fbf681879 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:20.829493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d08214-28fe-4683-b911-456fbf681879 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:20.847047 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1267/2561] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:12:20.851457 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613c451-c480-4dc9-9c4a-1c7c84f7b034 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:20.861255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613c451-c480-4dc9-9c4a-1c7c84f7b034 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:20.866930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc73d0-d3b5-4c9f-ba25-d21d70752528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:20.867595 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc73d0-d3b5-4c9f-ba25-d21d70752528 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:20.867894 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc73d0-d3b5-4c9f-ba25-d21d70752528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:20.868155 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc73d0-d3b5-4c9f-ba25-d21d70752528 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:20.868903 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc73d0-d3b5-4c9f-ba25-d21d70752528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:20.870845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc73d0-d3b5-4c9f-ba25-d21d70752528 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:20.926091 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1295/2562] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:12:20 2022] POST /identity/v3/projects => generated 359 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:20.927166 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d08214-28fe-4683-b911-456fbf681879 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:20.939022 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d08214-28fe-4683-b911-456fbf681879 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:21.009696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b75426-3373-4d48-bc2d-e9733a158a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:21.010389 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b75426-3373-4d48-bc2d-e9733a158a79 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:21.010730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b75426-3373-4d48-bc2d-e9733a158a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:21.011198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b75426-3373-4d48-bc2d-e9733a158a79 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:21.011714 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b75426-3373-4d48-bc2d-e9733a158a79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:21.014302 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b75426-3373-4d48-bc2d-e9733a158a79 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:21.038367 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1268/2563] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:12:20 2022] POST /identity/v3/users => generated 280 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:21.042625 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc73d0-d3b5-4c9f-ba25-d21d70752528 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:21.052976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc73d0-d3b5-4c9f-ba25-d21d70752528 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:21.057579 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a4cd6-beea-489b-a001-67ba5b975473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:21.058225 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a4cd6-beea-489b-a001-67ba5b975473 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:21.058504 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a4cd6-beea-489b-a001-67ba5b975473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:21.058777 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a4cd6-beea-489b-a001-67ba5b975473 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:21.059495 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a4cd6-beea-489b-a001-67ba5b975473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:21.061323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a4cd6-beea-489b-a001-67ba5b975473 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:21.072171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1296/2564] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:21.076839 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b75426-3373-4d48-bc2d-e9733a158a79 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:21.086857 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b75426-3373-4d48-bc2d-e9733a158a79 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:21.092463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a238ae2-1290-4b05-8be2-7c03ae7fa623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:21.093172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a238ae2-1290-4b05-8be2-7c03ae7fa623 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:21.093592 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a238ae2-1290-4b05-8be2-7c03ae7fa623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:21.093870 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a238ae2-1290-4b05-8be2-7c03ae7fa623 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:21.094598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a238ae2-1290-4b05-8be2-7c03ae7fa623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:21.096228 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a238ae2-1290-4b05-8be2-7c03ae7fa623 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:21.107084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1269/2565] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:21.111482 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a4cd6-beea-489b-a001-67ba5b975473 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:21.122872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a4cd6-beea-489b-a001-67ba5b975473 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:21.128165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac431b21-9c70-44ad-ac32-47f6d198c048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:21.129185 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac431b21-9c70-44ad-ac32-47f6d198c048 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:21.129487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac431b21-9c70-44ad-ac32-47f6d198c048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:21.129750 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac431b21-9c70-44ad-ac32-47f6d198c048 admin admin] PATH_INFO: `/v3/projects/a574b3c33b9e496488c74ddde17c6f48/users/a82caa73a25b405492d137ef7c4929a5/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:21.146321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac431b21-9c70-44ad-ac32-47f6d198c048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a574b3c33b9e496488c74ddde17c6f48, user_id=a82caa73a25b405492d137ef7c4929a5, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:21.149065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac431b21-9c70-44ad-ac32-47f6d198c048 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:21.162339 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1297/2566] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:12:21 2022] PUT /identity/v3/projects/a574b3c33b9e496488c74ddde17c6f48/users/a82caa73a25b405492d137ef7c4929a5/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:12:21.174271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4048b8-979f-4bfb-848a-33fa5f253082 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:21.174914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4048b8-979f-4bfb-848a-33fa5f253082 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:21.174914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4048b8-979f-4bfb-848a-33fa5f253082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:21.205757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-7e4048b8-979f-4bfb-848a-33fa5f253082 None None] MFA Rules not processed for user `a82caa73a25b405492d137ef7c4929a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:21.206981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-7e4048b8-979f-4bfb-848a-33fa5f253082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:21.272096 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1270/2567] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:21 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:21.304134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac431b21-9c70-44ad-ac32-47f6d198c048 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:21.305934 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ac431b21-9c70-44ad-ac32-47f6d198c048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:21.364331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac431b21-9c70-44ad-ac32-47f6d198c048 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:21.368075 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6f6c9-f729-4c02-a726-61d8f97cc2f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:21.368729 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6f6c9-f729-4c02-a726-61d8f97cc2f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:21.369034 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6f6c9-f729-4c02-a726-61d8f97cc2f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:21.369320 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6f6c9-f729-4c02-a726-61d8f97cc2f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:21.378669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d6f6c9-f729-4c02-a726-61d8f97cc2f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:21.380114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d6f6c9-f729-4c02-a726-61d8f97cc2f0 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:21.390296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1298/2568] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:21 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:21.404114 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4048b8-979f-4bfb-848a-33fa5f253082 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:21.446671 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:21.469685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4048b8-979f-4bfb-848a-33fa5f253082 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:21.474322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8df82-04ad-4afa-a6f5-491a49926714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:21.475185 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8df82-04ad-4afa-a6f5-491a49926714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:21.475466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8df82-04ad-4afa-a6f5-491a49926714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:21.475770 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8df82-04ad-4afa-a6f5-491a49926714 admin admin] PATH_INFO: `/v3/users/a82caa73a25b405492d137ef7c4929a5` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:21.477066 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8df82-04ad-4afa-a6f5-491a49926714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a82caa73a25b405492d137ef7c4929a5)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:21.478970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8df82-04ad-4afa-a6f5-491a49926714 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:21.523201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b2e8df82-04ad-4afa-a6f5-491a49926714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a82caa73a25b405492d137ef7c4929a5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:21.529932 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b2e8df82-04ad-4afa-a6f5-491a49926714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a82caa73a25b405492d137ef7c4929a5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:21.537086 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b2e8df82-04ad-4afa-a6f5-491a49926714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a82caa73a25b405492d137ef7c4929a5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:21.537694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b2e8df82-04ad-4afa-a6f5-491a49926714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a82caa73a25b405492d137ef7c4929a5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:21.538296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b2e8df82-04ad-4afa-a6f5-491a49926714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a82caa73a25b405492d137ef7c4929a5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:21.556747 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1271/2569] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:12:21 2022] DELETE /identity/v3/users/a82caa73a25b405492d137ef7c4929a5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:21.561938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440dd2e6-5444-44ca-aaf1-29051d24b73c None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:21.562232 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440dd2e6-5444-44ca-aaf1-29051d24b73c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:21.562538 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440dd2e6-5444-44ca-aaf1-29051d24b73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:21.580089 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-440dd2e6-5444-44ca-aaf1-29051d24b73c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:21.581326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-440dd2e6-5444-44ca-aaf1-29051d24b73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:21.608926 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:21.649836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1299/2570] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:21.660192 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8df82-04ad-4afa-a6f5-491a49926714 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:21.661839 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b2e8df82-04ad-4afa-a6f5-491a49926714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:21.723488 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8df82-04ad-4afa-a6f5-491a49926714 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:21.727209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff2d3f-bb3f-488f-acd2-72dfee6c7007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:21.727855 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff2d3f-bb3f-488f-acd2-72dfee6c7007 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:21.728138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff2d3f-bb3f-488f-acd2-72dfee6c7007 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:21.728466 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff2d3f-bb3f-488f-acd2-72dfee6c7007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:21.738020 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff2d3f-bb3f-488f-acd2-72dfee6c7007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:21.739370 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff2d3f-bb3f-488f-acd2-72dfee6c7007 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:21.749580 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1272/2571] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:22.046109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440dd2e6-5444-44ca-aaf1-29051d24b73c None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:22.047846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-440dd2e6-5444-44ca-aaf1-29051d24b73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:22.065027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440dd2e6-5444-44ca-aaf1-29051d24b73c None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:22.071696 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad01790-3445-40a6-90cb-400a52e7e8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:22.072727 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad01790-3445-40a6-90cb-400a52e7e8d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:22.073121 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad01790-3445-40a6-90cb-400a52e7e8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:22.073469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad01790-3445-40a6-90cb-400a52e7e8d5 admin admin] PATH_INFO: `/v3/projects/a574b3c33b9e496488c74ddde17c6f48` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:22.075036 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad01790-3445-40a6-90cb-400a52e7e8d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a574b3c33b9e496488c74ddde17c6f48)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:22.077998 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad01790-3445-40a6-90cb-400a52e7e8d5 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:22.130838 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dad01790-3445-40a6-90cb-400a52e7e8d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a574b3c33b9e496488c74ddde17c6f48', 'request_id': 'req-dad01790-3445-40a6-90cb-400a52e7e8d5'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:22.137462 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dad01790-3445-40a6-90cb-400a52e7e8d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a574b3c33b9e496488c74ddde17c6f48', 'request_id': 'req-dad01790-3445-40a6-90cb-400a52e7e8d5'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:22.150321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dad01790-3445-40a6-90cb-400a52e7e8d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a574b3c33b9e496488c74ddde17c6f48', 'request_id': 'req-dad01790-3445-40a6-90cb-400a52e7e8d5'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:22.157123 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff2d3f-bb3f-488f-acd2-72dfee6c7007 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:22.158673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-4cff2d3f-bb3f-488f-acd2-72dfee6c7007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:22.160492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dad01790-3445-40a6-90cb-400a52e7e8d5 admin admin] The token cache is being invalidate because project a574b3c33b9e496488c74ddde17c6f48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:12:22.160818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dad01790-3445-40a6-90cb-400a52e7e8d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:22.161477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-dad01790-3445-40a6-90cb-400a52e7e8d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:22.163025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1300/2572] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:12:22 2022] DELETE /identity/v3/projects/a574b3c33b9e496488c74ddde17c6f48 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:22.168479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a4a56-809e-4aad-8b0e-a47ad1003f2a None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:22.169020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a4a56-809e-4aad-8b0e-a47ad1003f2a None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:22.169481 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a4a56-809e-4aad-8b0e-a47ad1003f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:22.171254 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff2d3f-bb3f-488f-acd2-72dfee6c7007 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:22.189974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-612a4a56-809e-4aad-8b0e-a47ad1003f2a None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:22.191380 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-612a4a56-809e-4aad-8b0e-a47ad1003f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:22.227808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc788ec4-430f-42a0-95d5-c0df96d9217c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:22.228453 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc788ec4-430f-42a0-95d5-c0df96d9217c service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:22.228771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc788ec4-430f-42a0-95d5-c0df96d9217c service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:22.229055 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc788ec4-430f-42a0-95d5-c0df96d9217c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:22.231847 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-cc788ec4-430f-42a0-95d5-c0df96d9217c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:22.246174 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:22.260613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1301/2573] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:22.270286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35505b26-4393-410f-bc1c-cc364ec13f87 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:22.270733 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35505b26-4393-410f-bc1c-cc364ec13f87 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:22.271026 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35505b26-4393-410f-bc1c-cc364ec13f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:22.290411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-35505b26-4393-410f-bc1c-cc364ec13f87 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:22.291723 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-35505b26-4393-410f-bc1c-cc364ec13f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:22.301112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1302/2574] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:12:22.305087 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35505b26-4393-410f-bc1c-cc364ec13f87 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:22.314172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc788ec4-430f-42a0-95d5-c0df96d9217c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:22.315540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc788ec4-430f-42a0-95d5-c0df96d9217c service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:22.315651 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35505b26-4393-410f-bc1c-cc364ec13f87 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:22.320782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463c3b9-cf46-4b8e-ae52-7972621f5291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:22.321433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463c3b9-cf46-4b8e-ae52-7972621f5291 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:22.321722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463c3b9-cf46-4b8e-ae52-7972621f5291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:22.322033 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463c3b9-cf46-4b8e-ae52-7972621f5291 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:22.322784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463c3b9-cf46-4b8e-ae52-7972621f5291 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:12:22.323096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463c3b9-cf46-4b8e-ae52-7972621f5291 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:22.324746 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463c3b9-cf46-4b8e-ae52-7972621f5291 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:22.326306 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1273/2575] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:12:22.342592 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1303/2576] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 22:12:22.343699 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc788ec4-430f-42a0-95d5-c0df96d9217c service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:22.353922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc788ec4-430f-42a0-95d5-c0df96d9217c service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:22.359428 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ccbe0-ca3c-4fed-b5ef-201fba16454b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:22.360121 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ccbe0-ca3c-4fed-b5ef-201fba16454b admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:22.360433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ccbe0-ca3c-4fed-b5ef-201fba16454b admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:22.360936 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ccbe0-ca3c-4fed-b5ef-201fba16454b admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:22.361713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ccbe0-ca3c-4fed-b5ef-201fba16454b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:22.363821 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ccbe0-ca3c-4fed-b5ef-201fba16454b admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:22.393271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1274/2577] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:12:22 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:12:22.397843 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463c3b9-cf46-4b8e-ae52-7972621f5291 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:22.413384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463c3b9-cf46-4b8e-ae52-7972621f5291 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:22.449409 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:22.477855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a080732-6e29-473e-b315-863626458395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:22.478540 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a080732-6e29-473e-b315-863626458395 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:22.478823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a080732-6e29-473e-b315-863626458395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:22.479106 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a080732-6e29-473e-b315-863626458395 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:22.479811 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a080732-6e29-473e-b315-863626458395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:22.482804 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a080732-6e29-473e-b315-863626458395 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:22.507865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1304/2578] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:12:22 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:22.509215 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ccbe0-ca3c-4fed-b5ef-201fba16454b admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:22.520558 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ccbe0-ca3c-4fed-b5ef-201fba16454b admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:22.525320 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f68e5e-f213-47b6-b9ae-e72c5a18c496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:22.525986 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f68e5e-f213-47b6-b9ae-e72c5a18c496 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:22.526268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f68e5e-f213-47b6-b9ae-e72c5a18c496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:22.526543 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f68e5e-f213-47b6-b9ae-e72c5a18c496 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:22.527269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f68e5e-f213-47b6-b9ae-e72c5a18c496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:22.529598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f68e5e-f213-47b6-b9ae-e72c5a18c496 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:22.542977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1275/2579] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:22.545984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a080732-6e29-473e-b315-863626458395 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:22.556689 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a080732-6e29-473e-b315-863626458395 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:22.562359 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda2d5c-c995-47c1-97a8-0a7334b242c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:22.563043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda2d5c-c995-47c1-97a8-0a7334b242c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:22.563342 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda2d5c-c995-47c1-97a8-0a7334b242c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:22.563625 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda2d5c-c995-47c1-97a8-0a7334b242c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:22.564468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda2d5c-c995-47c1-97a8-0a7334b242c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:22.566103 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda2d5c-c995-47c1-97a8-0a7334b242c3 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:22.579530 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1305/2580] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:22.581397 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f68e5e-f213-47b6-b9ae-e72c5a18c496 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:22.592310 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f68e5e-f213-47b6-b9ae-e72c5a18c496 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:22.596961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ddfc6-a9ae-4086-b42c-2ffd694f7ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:22.597849 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ddfc6-a9ae-4086-b42c-2ffd694f7ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:22.598314 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ddfc6-a9ae-4086-b42c-2ffd694f7ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:22.598588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ddfc6-a9ae-4086-b42c-2ffd694f7ab2 admin admin] PATH_INFO: `/v3/projects/8b7d0eff80b94c0ebca41255d2db7033/users/c1c817118e2c436da2ccc33675ce5122/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:22.614268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ddfc6-a9ae-4086-b42c-2ffd694f7ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b7d0eff80b94c0ebca41255d2db7033, user_id=c1c817118e2c436da2ccc33675ce5122, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:22.617269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ddfc6-a9ae-4086-b42c-2ffd694f7ab2 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:22.631194 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1276/2581] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:12:22 2022] PUT /identity/v3/projects/8b7d0eff80b94c0ebca41255d2db7033/users/c1c817118e2c436da2ccc33675ce5122/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:12:22.643034 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410fd7-e917-44e8-a9f7-ddb77a6fd7fc None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:22.643469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410fd7-e917-44e8-a9f7-ddb77a6fd7fc None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:22.643886 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410fd7-e917-44e8-a9f7-ddb77a6fd7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:22.674149 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-1a410fd7-e917-44e8-a9f7-ddb77a6fd7fc None None] MFA Rules not processed for user `c1c817118e2c436da2ccc33675ce5122`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:22.675417 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-1a410fd7-e917-44e8-a9f7-ddb77a6fd7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:22.741632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1306/2582] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:22.746260 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ddfc6-a9ae-4086-b42c-2ffd694f7ab2 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:22.815673 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ddfc6-a9ae-4086-b42c-2ffd694f7ab2 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:22.820808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd1b86-09d1-4670-86f4-39733891248b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:22.821499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd1b86-09d1-4670-86f4-39733891248b admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:22.821782 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd1b86-09d1-4670-86f4-39733891248b admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:22.822064 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd1b86-09d1-4670-86f4-39733891248b admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:22.822753 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd1b86-09d1-4670-86f4-39733891248b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:22.824646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd1b86-09d1-4670-86f4-39733891248b admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:22.851995 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1277/2583] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:12:22 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:22.857417 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a410fd7-e917-44e8-a9f7-ddb77a6fd7fc None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:22.871120 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a410fd7-e917-44e8-a9f7-ddb77a6fd7fc None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:22.935240 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43398aa-b3bc-42d5-9137-7ea972c38ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:22.935961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43398aa-b3bc-42d5-9137-7ea972c38ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:22.936290 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43398aa-b3bc-42d5-9137-7ea972c38ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:22.936760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43398aa-b3bc-42d5-9137-7ea972c38ed6 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:22.937407 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43398aa-b3bc-42d5-9137-7ea972c38ed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:22.939413 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43398aa-b3bc-42d5-9137-7ea972c38ed6 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:22.964700 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1307/2584] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:12:22 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:22.966966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd1b86-09d1-4670-86f4-39733891248b admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:22.978221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd1b86-09d1-4670-86f4-39733891248b admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:22.986196 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe895179-e0c6-4e3d-8e6a-339e052cfab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:22.986884 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe895179-e0c6-4e3d-8e6a-339e052cfab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:22.987167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe895179-e0c6-4e3d-8e6a-339e052cfab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:22.987442 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe895179-e0c6-4e3d-8e6a-339e052cfab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:22.988495 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe895179-e0c6-4e3d-8e6a-339e052cfab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:22.989956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe895179-e0c6-4e3d-8e6a-339e052cfab6 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:23.005501 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1278/2585] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:23.007182 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43398aa-b3bc-42d5-9137-7ea972c38ed6 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:23.028919 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43398aa-b3bc-42d5-9137-7ea972c38ed6 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:23.034200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6da06-26db-4147-9f3d-905ececc6052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:23.035103 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6da06-26db-4147-9f3d-905ececc6052 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:23.035462 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6da06-26db-4147-9f3d-905ececc6052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:23.035816 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6da06-26db-4147-9f3d-905ececc6052 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:23.036843 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6da06-26db-4147-9f3d-905ececc6052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:23.038721 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6da06-26db-4147-9f3d-905ececc6052 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:23.052882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1308/2586] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:23.054946 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe895179-e0c6-4e3d-8e6a-339e052cfab6 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:23.073540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe895179-e0c6-4e3d-8e6a-339e052cfab6 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:23.073540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13db3e-11b4-4c8e-b2f2-f3c7c6f2dc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:23.073540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13db3e-11b4-4c8e-b2f2-f3c7c6f2dc03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:23.073540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13db3e-11b4-4c8e-b2f2-f3c7c6f2dc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:23.073932 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13db3e-11b4-4c8e-b2f2-f3c7c6f2dc03 admin admin] PATH_INFO: `/v3/projects/72d3fd1a02e649ce8ee81b91059c11de/users/b3d37bb0de424f208ab703615c223534/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:23.089719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13db3e-11b4-4c8e-b2f2-f3c7c6f2dc03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d3fd1a02e649ce8ee81b91059c11de, user_id=b3d37bb0de424f208ab703615c223534, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:23.093222 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13db3e-11b4-4c8e-b2f2-f3c7c6f2dc03 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:23.114055 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1279/2587] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:12:23 2022] PUT /identity/v3/projects/72d3fd1a02e649ce8ee81b91059c11de/users/b3d37bb0de424f208ab703615c223534/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:23.119522 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49102a37-6984-41fa-b29f-5ba6e1617668 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:23.119819 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49102a37-6984-41fa-b29f-5ba6e1617668 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:23.120102 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49102a37-6984-41fa-b29f-5ba6e1617668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:23.161102 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-49102a37-6984-41fa-b29f-5ba6e1617668 None None] MFA Rules not processed for user `b3d37bb0de424f208ab703615c223534`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:23.163722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-49102a37-6984-41fa-b29f-5ba6e1617668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:23.177251 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:23.260096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1309/2588] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:23.267639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13db3e-11b4-4c8e-b2f2-f3c7c6f2dc03 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:23.273060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:23.351709 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13db3e-11b4-4c8e-b2f2-f3c7c6f2dc03 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:23.358068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeeffd9-b9dd-47f6-85eb-680a4487ca2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:23.359431 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeeffd9-b9dd-47f6-85eb-680a4487ca2e admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:23.359431 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeeffd9-b9dd-47f6-85eb-680a4487ca2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:23.359648 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeeffd9-b9dd-47f6-85eb-680a4487ca2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:23.360476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeeffd9-b9dd-47f6-85eb-680a4487ca2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:23.362879 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeeffd9-b9dd-47f6-85eb-680a4487ca2e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:23.407148 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1280/2589] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:12:23 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:23.411440 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49102a37-6984-41fa-b29f-5ba6e1617668 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:23.426895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49102a37-6984-41fa-b29f-5ba6e1617668 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:23.493763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768aa7e-754b-4f9c-8f43-d97eb265eb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:23.494590 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768aa7e-754b-4f9c-8f43-d97eb265eb85 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:23.494948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768aa7e-754b-4f9c-8f43-d97eb265eb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:23.495300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768aa7e-754b-4f9c-8f43-d97eb265eb85 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:23.496475 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768aa7e-754b-4f9c-8f43-d97eb265eb85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:23.498279 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768aa7e-754b-4f9c-8f43-d97eb265eb85 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:23.522434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1310/2590] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:12:23 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:23.523865 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeeffd9-b9dd-47f6-85eb-680a4487ca2e admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:23.537014 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeeffd9-b9dd-47f6-85eb-680a4487ca2e admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:23.543452 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f67e8c-2c25-406e-9023-e18390e75bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:23.544669 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f67e8c-2c25-406e-9023-e18390e75bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:23.544988 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f67e8c-2c25-406e-9023-e18390e75bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:23.545269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f67e8c-2c25-406e-9023-e18390e75bb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:23.546012 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f67e8c-2c25-406e-9023-e18390e75bb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:23.547806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f67e8c-2c25-406e-9023-e18390e75bb6 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:23.559800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1281/2591] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:23.564258 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768aa7e-754b-4f9c-8f43-d97eb265eb85 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:23.575674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768aa7e-754b-4f9c-8f43-d97eb265eb85 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:23.582823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42384ef4-a9af-47b7-a543-45dfaaf3ea1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:23.583826 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42384ef4-a9af-47b7-a543-45dfaaf3ea1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:23.584209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42384ef4-a9af-47b7-a543-45dfaaf3ea1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:23.584463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42384ef4-a9af-47b7-a543-45dfaaf3ea1c admin admin] PATH_INFO: `/v3/projects/f3b9a20ecb9c49a1b0d574daf4861dca/users/bc2376aa8b1f42f987d51fe92a3df439/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:23.603773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42384ef4-a9af-47b7-a543-45dfaaf3ea1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3b9a20ecb9c49a1b0d574daf4861dca, user_id=bc2376aa8b1f42f987d51fe92a3df439, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:23.607520 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42384ef4-a9af-47b7-a543-45dfaaf3ea1c admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:23.625997 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1311/2592] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:12:23 2022] PUT /identity/v3/projects/f3b9a20ecb9c49a1b0d574daf4861dca/users/bc2376aa8b1f42f987d51fe92a3df439/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:23.630107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f67e8c-2c25-406e-9023-e18390e75bb6 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:23.642243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f67e8c-2c25-406e-9023-e18390e75bb6 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:23.710264 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a0f7e-4fcb-4a33-807e-bcfd2f325e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:23.711033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a0f7e-4fcb-4a33-807e-bcfd2f325e30 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:23.711324 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a0f7e-4fcb-4a33-807e-bcfd2f325e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:23.711609 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a0f7e-4fcb-4a33-807e-bcfd2f325e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:23.712349 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a0f7e-4fcb-4a33-807e-bcfd2f325e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:23.714497 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a0f7e-4fcb-4a33-807e-bcfd2f325e30 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:23.725136 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1282/2593] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:23.729329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42384ef4-a9af-47b7-a543-45dfaaf3ea1c admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:23.740607 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42384ef4-a9af-47b7-a543-45dfaaf3ea1c admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:23.745463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf262a-0e95-4239-9918-10edb41cb4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:23.746496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf262a-0e95-4239-9918-10edb41cb4e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:23.746993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf262a-0e95-4239-9918-10edb41cb4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:23.747844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf262a-0e95-4239-9918-10edb41cb4e5 admin admin] PATH_INFO: `/v3/projects/f3b9a20ecb9c49a1b0d574daf4861dca/users/bc2376aa8b1f42f987d51fe92a3df439/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:23.750009 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf262a-0e95-4239-9918-10edb41cb4e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3b9a20ecb9c49a1b0d574daf4861dca, user_id=bc2376aa8b1f42f987d51fe92a3df439, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:23.752678 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf262a-0e95-4239-9918-10edb41cb4e5 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:23.764598 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1312/2594] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:12:23 2022] PUT /identity/v3/projects/f3b9a20ecb9c49a1b0d574daf4861dca/users/bc2376aa8b1f42f987d51fe92a3df439/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:23.775329 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452f208-b585-4e99-87bb-e83355ecf0c0 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:23.775674 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452f208-b585-4e99-87bb-e83355ecf0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:23.776213 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452f208-b585-4e99-87bb-e83355ecf0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:23.806424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-0452f208-b585-4e99-87bb-e83355ecf0c0 None None] MFA Rules not processed for user `bc2376aa8b1f42f987d51fe92a3df439`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:23.807435 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0452f208-b585-4e99-87bb-e83355ecf0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:23.874904 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:23.881617 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1283/2595] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:23.893904 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf262a-0e95-4239-9918-10edb41cb4e5 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:23.895752 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e1bf262a-0e95-4239-9918-10edb41cb4e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:23.956043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf262a-0e95-4239-9918-10edb41cb4e5 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:23.959742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbabfa0-6454-4e62-815f-553ba6c0c6af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:23.960395 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbabfa0-6454-4e62-815f-553ba6c0c6af service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:23.960683 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbabfa0-6454-4e62-815f-553ba6c0c6af service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:23.961053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbabfa0-6454-4e62-815f-553ba6c0c6af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:23.980851 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:24.023192 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbabfa0-6454-4e62-815f-553ba6c0c6af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:24.024548 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbabfa0-6454-4e62-815f-553ba6c0c6af service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:24.035501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1313/2596] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:25.682163 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452f208-b585-4e99-87bb-e83355ecf0c0 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:25.683771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0452f208-b585-4e99-87bb-e83355ecf0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:25.705624 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452f208-b585-4e99-87bb-e83355ecf0c0 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:25.714269 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf0a18-9d33-4d1b-acb0-54bee23f95c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:25.715032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf0a18-9d33-4d1b-acb0-54bee23f95c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:25.715611 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf0a18-9d33-4d1b-acb0-54bee23f95c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:25.716076 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf0a18-9d33-4d1b-acb0-54bee23f95c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:25.847616 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf0a18-9d33-4d1b-acb0-54bee23f95c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:25.852201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf0a18-9d33-4d1b-acb0-54bee23f95c2 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:25.880654 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1284/2597] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:25.994542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbabfa0-6454-4e62-815f-553ba6c0c6af service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:25.996119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-dbbabfa0-6454-4e62-815f-553ba6c0c6af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:26.095354 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbabfa0-6454-4e62-815f-553ba6c0c6af service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:26.100736 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8f905-080f-465b-8c1c-98661761d9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:26.101472 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8f905-080f-465b-8c1c-98661761d9b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:26.101817 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8f905-080f-465b-8c1c-98661761d9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:26.102167 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8f905-080f-465b-8c1c-98661761d9b5 admin admin] PATH_INFO: `/v3/users/4a746b42824c4d13a1d0b0f2c5831259` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:26.103495 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8f905-080f-465b-8c1c-98661761d9b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a746b42824c4d13a1d0b0f2c5831259)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:26.105492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8f905-080f-465b-8c1c-98661761d9b5 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:26.196538 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c9e8f905-080f-465b-8c1c-98661761d9b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a746b42824c4d13a1d0b0f2c5831259'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:26.214900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c9e8f905-080f-465b-8c1c-98661761d9b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a746b42824c4d13a1d0b0f2c5831259'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:26.223862 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c9e8f905-080f-465b-8c1c-98661761d9b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a746b42824c4d13a1d0b0f2c5831259'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:26.238587 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c9e8f905-080f-465b-8c1c-98661761d9b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a746b42824c4d13a1d0b0f2c5831259'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:26.239438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c9e8f905-080f-465b-8c1c-98661761d9b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a746b42824c4d13a1d0b0f2c5831259'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:26.247802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1314/2598] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:12:25 2022] DELETE /identity/v3/users/4a746b42824c4d13a1d0b0f2c5831259 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:26.475809 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf0a18-9d33-4d1b-acb0-54bee23f95c2 service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:26.490789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf0a18-9d33-4d1b-acb0-54bee23f95c2 service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:26.564663 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94343458-c623-4d97-94ab-1fc47dd6b6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:26.565702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94343458-c623-4d97-94ab-1fc47dd6b6d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:26.566057 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94343458-c623-4d97-94ab-1fc47dd6b6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:26.566453 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94343458-c623-4d97-94ab-1fc47dd6b6d5 admin admin] PATH_INFO: `/v3/projects/807d2d1b808448bca8aaf30389c8c571` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:26.566855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8f905-080f-465b-8c1c-98661761d9b5 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:26.567925 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94343458-c623-4d97-94ab-1fc47dd6b6d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=807d2d1b808448bca8aaf30389c8c571)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:26.568887 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c9e8f905-080f-465b-8c1c-98661761d9b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:26.570341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94343458-c623-4d97-94ab-1fc47dd6b6d5 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:26.624703 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-94343458-c623-4d97-94ab-1fc47dd6b6d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '807d2d1b808448bca8aaf30389c8c571', 'request_id': 'req-94343458-c623-4d97-94ab-1fc47dd6b6d5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:26.627066 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:26.633404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-94343458-c623-4d97-94ab-1fc47dd6b6d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '807d2d1b808448bca8aaf30389c8c571', 'request_id': 'req-94343458-c623-4d97-94ab-1fc47dd6b6d5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:26.644456 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-94343458-c623-4d97-94ab-1fc47dd6b6d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '807d2d1b808448bca8aaf30389c8c571', 'request_id': 'req-94343458-c623-4d97-94ab-1fc47dd6b6d5'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:26.649447 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8f905-080f-465b-8c1c-98661761d9b5 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:26.658566 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-94343458-c623-4d97-94ab-1fc47dd6b6d5 admin admin] The token cache is being invalidate because project 807d2d1b808448bca8aaf30389c8c571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:12:26.659026 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-94343458-c623-4d97-94ab-1fc47dd6b6d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:26.659938 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-94343458-c623-4d97-94ab-1fc47dd6b6d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:26.662644 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1285/2599] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:12:26 2022] DELETE /identity/v3/projects/807d2d1b808448bca8aaf30389c8c571 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:26.667491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d84c51-6ceb-4acf-b26c-662a0ff93308 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:26.667843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d84c51-6ceb-4acf-b26c-662a0ff93308 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:26.668183 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d84c51-6ceb-4acf-b26c-662a0ff93308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:26.691073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-b7d84c51-6ceb-4acf-b26c-662a0ff93308 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:26.692447 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b7d84c51-6ceb-4acf-b26c-662a0ff93308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:26.712340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6a19c-a79a-4482-9c6d-ff4fab312cf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:26.713832 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6a19c-a79a-4482-9c6d-ff4fab312cf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:26.714441 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6a19c-a79a-4482-9c6d-ff4fab312cf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:26.714863 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:26.715315 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6a19c-a79a-4482-9c6d-ff4fab312cf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:26.717647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-91a6a19c-a79a-4482-9c6d-ff4fab312cf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:26.767368 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1286/2600] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:12:26.775376 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f00cff-c1c7-4cda-8493-33414684368f None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:26.775752 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f00cff-c1c7-4cda-8493-33414684368f None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:26.775987 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f00cff-c1c7-4cda-8493-33414684368f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:26.796171 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-39f00cff-c1c7-4cda-8493-33414684368f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:26.797435 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-39f00cff-c1c7-4cda-8493-33414684368f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:26.804165 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a6a19c-a79a-4482-9c6d-ff4fab312cf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:26.805810 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a6a19c-a79a-4482-9c6d-ff4fab312cf6 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:26.806514 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1287/2601] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:12:26.811761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f00cff-c1c7-4cda-8493-33414684368f None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:26.822722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1315/2602] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:12:26.823592 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f00cff-c1c7-4cda-8493-33414684368f None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:26.829459 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e8a5b-47e9-424b-b3be-538f32f5ddbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:26.829715 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e8a5b-47e9-424b-b3be-538f32f5ddbd admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:26.829992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e8a5b-47e9-424b-b3be-538f32f5ddbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:26.830332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e8a5b-47e9-424b-b3be-538f32f5ddbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:26.831060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e8a5b-47e9-424b-b3be-538f32f5ddbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:12:26.831370 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e8a5b-47e9-424b-b3be-538f32f5ddbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:26.832986 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e8a5b-47e9-424b-b3be-538f32f5ddbd admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:26.847954 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1288/2603] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:12:26.852191 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6a19c-a79a-4482-9c6d-ff4fab312cf6 service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:26.865164 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6a19c-a79a-4482-9c6d-ff4fab312cf6 service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:26.870752 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6b868-eed1-4abc-9a44-ceecd69c047c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:26.871430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6b868-eed1-4abc-9a44-ceecd69c047c admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:26.872356 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6b868-eed1-4abc-9a44-ceecd69c047c admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:26.872641 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6b868-eed1-4abc-9a44-ceecd69c047c admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:26.873450 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6b868-eed1-4abc-9a44-ceecd69c047c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:26.877822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6b868-eed1-4abc-9a44-ceecd69c047c admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:26.913951 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1316/2604] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:12:26 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:26.918180 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e8a5b-47e9-424b-b3be-538f32f5ddbd admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:26.932592 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e8a5b-47e9-424b-b3be-538f32f5ddbd admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:27.020499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae4f27-0997-4909-809b-4f6c98980268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:27.021297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae4f27-0997-4909-809b-4f6c98980268 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:27.021730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae4f27-0997-4909-809b-4f6c98980268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:27.022089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae4f27-0997-4909-809b-4f6c98980268 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:27.023751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae4f27-0997-4909-809b-4f6c98980268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:27.026947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae4f27-0997-4909-809b-4f6c98980268 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:27.048616 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1289/2605] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:12:26 2022] POST /identity/v3/users => generated 289 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:27.053283 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6b868-eed1-4abc-9a44-ceecd69c047c admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:27.065572 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6b868-eed1-4abc-9a44-ceecd69c047c admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:27.070969 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc480d-ad14-47cc-b3b9-b08ae1e67441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:27.071786 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc480d-ad14-47cc-b3b9-b08ae1e67441 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:27.072142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc480d-ad14-47cc-b3b9-b08ae1e67441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:27.072496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc480d-ad14-47cc-b3b9-b08ae1e67441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:27.073414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc480d-ad14-47cc-b3b9-b08ae1e67441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:27.075270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc480d-ad14-47cc-b3b9-b08ae1e67441 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:27.087312 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1317/2606] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:27.093133 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae4f27-0997-4909-809b-4f6c98980268 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:27.104695 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae4f27-0997-4909-809b-4f6c98980268 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:27.110107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fea29d-16e0-429f-9d31-829f4fe51d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:27.111181 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fea29d-16e0-429f-9d31-829f4fe51d92 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:27.111361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fea29d-16e0-429f-9d31-829f4fe51d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:27.111642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fea29d-16e0-429f-9d31-829f4fe51d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:27.112373 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fea29d-16e0-429f-9d31-829f4fe51d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:27.114028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fea29d-16e0-429f-9d31-829f4fe51d92 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:27.127651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1290/2607] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:27.131342 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc480d-ad14-47cc-b3b9-b08ae1e67441 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:27.143714 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc480d-ad14-47cc-b3b9-b08ae1e67441 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:27.148870 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1c842-ef13-49f7-bee7-8bcd17cb5b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:27.149839 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1c842-ef13-49f7-bee7-8bcd17cb5b2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:27.150329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1c842-ef13-49f7-bee7-8bcd17cb5b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:27.150606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1c842-ef13-49f7-bee7-8bcd17cb5b2d admin admin] PATH_INFO: `/v3/projects/b80ca69f6cd7402cb644c7721015ebc8/users/2d0d387beaf842c3b5978210e504fbed/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:27.169995 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1c842-ef13-49f7-bee7-8bcd17cb5b2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b80ca69f6cd7402cb644c7721015ebc8, user_id=2d0d387beaf842c3b5978210e504fbed, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:27.173738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1c842-ef13-49f7-bee7-8bcd17cb5b2d admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:27.189414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1318/2608] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:12:27 2022] PUT /identity/v3/projects/b80ca69f6cd7402cb644c7721015ebc8/users/2d0d387beaf842c3b5978210e504fbed/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:27.195591 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cd65e-ffbd-4312-ad85-1cccc01ffb47 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:27.195984 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cd65e-ffbd-4312-ad85-1cccc01ffb47 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:27.196395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cd65e-ffbd-4312-ad85-1cccc01ffb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:27.217015 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-571cd65e-ffbd-4312-ad85-1cccc01ffb47 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:27.218375 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-571cd65e-ffbd-4312-ad85-1cccc01ffb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:27.278470 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1291/2609] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:27.288271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1c842-ef13-49f7-bee7-8bcd17cb5b2d admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:27.290356 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-36b1c842-ef13-49f7-bee7-8bcd17cb5b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:27.354576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:27.367043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1c842-ef13-49f7-bee7-8bcd17cb5b2d admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:27.370950 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34cc36-bcdb-4e2f-ba87-8cde6a1aa216 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:27.372251 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34cc36-bcdb-4e2f-ba87-8cde6a1aa216 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:27.372985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34cc36-bcdb-4e2f-ba87-8cde6a1aa216 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:27.373413 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34cc36-bcdb-4e2f-ba87-8cde6a1aa216 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:27.384488 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34cc36-bcdb-4e2f-ba87-8cde6a1aa216 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:27.385967 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34cc36-bcdb-4e2f-ba87-8cde6a1aa216 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:27.396921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1319/2610] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:30.600254 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cd65e-ffbd-4312-ad85-1cccc01ffb47 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:30.602197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-571cd65e-ffbd-4312-ad85-1cccc01ffb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:30.627767 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cd65e-ffbd-4312-ad85-1cccc01ffb47 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:30.636307 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75e046-f5b2-46d2-a18c-b0725bb1208f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:30.637770 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75e046-f5b2-46d2-a18c-b0725bb1208f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:30.638281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75e046-f5b2-46d2-a18c-b0725bb1208f admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:30.638728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75e046-f5b2-46d2-a18c-b0725bb1208f admin admin] PATH_INFO: `/v3/users/6fc8ec2fda70494c8a705f17f40deb40` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:30.640816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad75e046-f5b2-46d2-a18c-b0725bb1208f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fc8ec2fda70494c8a705f17f40deb40)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:30.642990 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad75e046-f5b2-46d2-a18c-b0725bb1208f admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:30.670204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:30.706366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ad75e046-f5b2-46d2-a18c-b0725bb1208f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fc8ec2fda70494c8a705f17f40deb40'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:30.714919 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ad75e046-f5b2-46d2-a18c-b0725bb1208f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fc8ec2fda70494c8a705f17f40deb40'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:30.723850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ad75e046-f5b2-46d2-a18c-b0725bb1208f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fc8ec2fda70494c8a705f17f40deb40'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:30.724866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ad75e046-f5b2-46d2-a18c-b0725bb1208f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fc8ec2fda70494c8a705f17f40deb40'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:30.726126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ad75e046-f5b2-46d2-a18c-b0725bb1208f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fc8ec2fda70494c8a705f17f40deb40'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:30.755192 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1292/2611] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:12:30 2022] DELETE /identity/v3/users/6fc8ec2fda70494c8a705f17f40deb40 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:30.759652 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ffa79-1cbe-457a-9ad0-c299fe93c7b1 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:30.760163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ffa79-1cbe-457a-9ad0-c299fe93c7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:30.760265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ffa79-1cbe-457a-9ad0-c299fe93c7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:30.778732 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-b48ffa79-1cbe-457a-9ad0-c299fe93c7b1 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:30.780263 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b48ffa79-1cbe-457a-9ad0-c299fe93c7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:30.856807 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1320/2612] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:30.864292 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75e046-f5b2-46d2-a18c-b0725bb1208f admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:30.865789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ad75e046-f5b2-46d2-a18c-b0725bb1208f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:30.980108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ffa79-1cbe-457a-9ad0-c299fe93c7b1 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:30.982505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b48ffa79-1cbe-457a-9ad0-c299fe93c7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:31.006411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ffa79-1cbe-457a-9ad0-c299fe93c7b1 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:31.008548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75e046-f5b2-46d2-a18c-b0725bb1208f admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:31.015571 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789335a5-298f-4ef5-a5ed-c597c85a4f08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:31.016033 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789335a5-298f-4ef5-a5ed-c597c85a4f08 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:31.016319 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789335a5-298f-4ef5-a5ed-c597c85a4f08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:31.016597 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789335a5-298f-4ef5-a5ed-c597c85a4f08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:31.022572 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbc09c-bdac-4786-9512-a847138bd528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:31.025593 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbc09c-bdac-4786-9512-a847138bd528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:31.028947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789335a5-298f-4ef5-a5ed-c597c85a4f08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:31.030933 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbc09c-bdac-4786-9512-a847138bd528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:31.031076 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789335a5-298f-4ef5-a5ed-c597c85a4f08 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:31.031565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbc09c-bdac-4786-9512-a847138bd528 admin admin] PATH_INFO: `/v3/users/c1c817118e2c436da2ccc33675ce5122` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:31.033322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbc09c-bdac-4786-9512-a847138bd528 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c817118e2c436da2ccc33675ce5122)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:31.035777 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbc09c-bdac-4786-9512-a847138bd528 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:31.063025 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1293/2613] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:31.131187 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-00bbc09c-bdac-4786-9512-a847138bd528 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c817118e2c436da2ccc33675ce5122'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:31.152746 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-00bbc09c-bdac-4786-9512-a847138bd528 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c817118e2c436da2ccc33675ce5122'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:31.167315 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-00bbc09c-bdac-4786-9512-a847138bd528 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c817118e2c436da2ccc33675ce5122'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:31.189123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-00bbc09c-bdac-4786-9512-a847138bd528 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c817118e2c436da2ccc33675ce5122'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:31.189796 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-00bbc09c-bdac-4786-9512-a847138bd528 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c817118e2c436da2ccc33675ce5122'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:31.199208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1321/2614] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:12:30 2022] DELETE /identity/v3/users/c1c817118e2c436da2ccc33675ce5122 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:12:31.204928 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c07e9a-f8c2-46c4-8f5a-ffca24a75cd0 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:31.205245 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c07e9a-f8c2-46c4-8f5a-ffca24a75cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:31.205507 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c07e9a-f8c2-46c4-8f5a-ffca24a75cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:31.226859 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-e4c07e9a-f8c2-46c4-8f5a-ffca24a75cd0 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:31.228905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e4c07e9a-f8c2-46c4-8f5a-ffca24a75cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:31.308259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1294/2615] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:31.319319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbc09c-bdac-4786-9512-a847138bd528 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:31.320871 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-00bbc09c-bdac-4786-9512-a847138bd528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:31.395616 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbc09c-bdac-4786-9512-a847138bd528 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:31.400676 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97350c51-0837-4400-a6f5-55bf530cf5fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:31.401501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97350c51-0837-4400-a6f5-55bf530cf5fe service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:31.401931 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97350c51-0837-4400-a6f5-55bf530cf5fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:31.402498 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97350c51-0837-4400-a6f5-55bf530cf5fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:31.419900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97350c51-0837-4400-a6f5-55bf530cf5fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:31.421105 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97350c51-0837-4400-a6f5-55bf530cf5fe service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:31.434787 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1322/2616] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:31.613395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c07e9a-f8c2-46c4-8f5a-ffca24a75cd0 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:31.615067 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e4c07e9a-f8c2-46c4-8f5a-ffca24a75cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:31.636483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c07e9a-f8c2-46c4-8f5a-ffca24a75cd0 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:31.642674 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e874487-f1d0-4d7d-b58f-269de0a6c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:31.644602 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e874487-f1d0-4d7d-b58f-269de0a6c3ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:31.645357 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e874487-f1d0-4d7d-b58f-269de0a6c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:31.647097 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e874487-f1d0-4d7d-b58f-269de0a6c3ad admin admin] PATH_INFO: `/v3/projects/6a96f4f546604069a96d7b4fa8b64900` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:31.649861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e874487-f1d0-4d7d-b58f-269de0a6c3ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a96f4f546604069a96d7b4fa8b64900)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:31.655871 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e874487-f1d0-4d7d-b58f-269de0a6c3ad admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:31.724113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:31.745249 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4e874487-f1d0-4d7d-b58f-269de0a6c3ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a96f4f546604069a96d7b4fa8b64900', 'request_id': 'req-4e874487-f1d0-4d7d-b58f-269de0a6c3ad'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:31.756821 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4e874487-f1d0-4d7d-b58f-269de0a6c3ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a96f4f546604069a96d7b4fa8b64900', 'request_id': 'req-4e874487-f1d0-4d7d-b58f-269de0a6c3ad'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:31.770215 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4e874487-f1d0-4d7d-b58f-269de0a6c3ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a96f4f546604069a96d7b4fa8b64900', 'request_id': 'req-4e874487-f1d0-4d7d-b58f-269de0a6c3ad'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:31.807259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4e874487-f1d0-4d7d-b58f-269de0a6c3ad admin admin] The token cache is being invalidate because project 6a96f4f546604069a96d7b4fa8b64900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:12:31.812940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4e874487-f1d0-4d7d-b58f-269de0a6c3ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:31.814798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-4e874487-f1d0-4d7d-b58f-269de0a6c3ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:31.820633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1295/2617] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:12:31 2022] DELETE /identity/v3/projects/6a96f4f546604069a96d7b4fa8b64900 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:31.833422 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97350c51-0837-4400-a6f5-55bf530cf5fe service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:31.836117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-97350c51-0837-4400-a6f5-55bf530cf5fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:31.858116 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:31.863726 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97350c51-0837-4400-a6f5-55bf530cf5fe service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:31.945158 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f41ee-01ef-49cc-813b-370f18719d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:31.946511 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f41ee-01ef-49cc-813b-370f18719d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:31.946956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f41ee-01ef-49cc-813b-370f18719d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:31.947455 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f41ee-01ef-49cc-813b-370f18719d4f admin admin] PATH_INFO: `/v3/projects/8b7d0eff80b94c0ebca41255d2db7033` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:31.949070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f41ee-01ef-49cc-813b-370f18719d4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b7d0eff80b94c0ebca41255d2db7033)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:31.952207 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f41ee-01ef-49cc-813b-370f18719d4f admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:32.055044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-658f41ee-01ef-49cc-813b-370f18719d4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b7d0eff80b94c0ebca41255d2db7033', 'request_id': 'req-658f41ee-01ef-49cc-813b-370f18719d4f'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:32.072188 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-658f41ee-01ef-49cc-813b-370f18719d4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b7d0eff80b94c0ebca41255d2db7033', 'request_id': 'req-658f41ee-01ef-49cc-813b-370f18719d4f'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:32.083727 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-658f41ee-01ef-49cc-813b-370f18719d4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b7d0eff80b94c0ebca41255d2db7033', 'request_id': 'req-658f41ee-01ef-49cc-813b-370f18719d4f'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:32.096201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1413f-9d29-4ca0-9ae4-0d569d6c01e8 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:32.096480 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1413f-9d29-4ca0-9ae4-0d569d6c01e8 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:32.098092 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1413f-9d29-4ca0-9ae4-0d569d6c01e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:32.101206 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-658f41ee-01ef-49cc-813b-370f18719d4f admin admin] The token cache is being invalidate because project 8b7d0eff80b94c0ebca41255d2db7033 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:12:32.101704 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-658f41ee-01ef-49cc-813b-370f18719d4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:32.102489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-658f41ee-01ef-49cc-813b-370f18719d4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:32.104588 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1323/2618] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:12:31 2022] DELETE /identity/v3/projects/8b7d0eff80b94c0ebca41255d2db7033 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:32.109213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f41ee-01ef-49cc-813b-370f18719d4f admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:32.111042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-658f41ee-01ef-49cc-813b-370f18719d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:32.162252 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-d6d1413f-9d29-4ca0-9ae4-0d569d6c01e8 None None] MFA Rules not processed for user `2d0d387beaf842c3b5978210e504fbed`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:32.165237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d6d1413f-9d29-4ca0-9ae4-0d569d6c01e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:32.196940 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f41ee-01ef-49cc-813b-370f18719d4f admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:32.202605 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8516545-0dda-4bce-9a4e-e8beeedacbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:32.203489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8516545-0dda-4bce-9a4e-e8beeedacbee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:32.203818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8516545-0dda-4bce-9a4e-e8beeedacbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:32.204100 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8516545-0dda-4bce-9a4e-e8beeedacbee admin admin] PATH_INFO: `/v3/users/b3d37bb0de424f208ab703615c223534` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:32.205474 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8516545-0dda-4bce-9a4e-e8beeedacbee admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3d37bb0de424f208ab703615c223534)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:32.207387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8516545-0dda-4bce-9a4e-e8beeedacbee admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:32.248263 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1296/2619] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:32.263865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:32.264924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1413f-9d29-4ca0-9ae4-0d569d6c01e8 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:32.266418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d6d1413f-9d29-4ca0-9ae4-0d569d6c01e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:32.287065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f8516545-0dda-4bce-9a4e-e8beeedacbee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3d37bb0de424f208ab703615c223534'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:32.305417 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f8516545-0dda-4bce-9a4e-e8beeedacbee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3d37bb0de424f208ab703615c223534'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:32.321319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f8516545-0dda-4bce-9a4e-e8beeedacbee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3d37bb0de424f208ab703615c223534'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:32.358312 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f8516545-0dda-4bce-9a4e-e8beeedacbee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3d37bb0de424f208ab703615c223534'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:32.359673 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f8516545-0dda-4bce-9a4e-e8beeedacbee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3d37bb0de424f208ab703615c223534'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:32.368375 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1324/2620] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:12:32 2022] DELETE /identity/v3/users/b3d37bb0de424f208ab703615c223534 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:12:32.396082 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1413f-9d29-4ca0-9ae4-0d569d6c01e8 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:32.473316 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:32.505249 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a961e07-eb62-4c9c-ad04-dccd531013cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:32.506077 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a961e07-eb62-4c9c-ad04-dccd531013cf service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:32.506471 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a961e07-eb62-4c9c-ad04-dccd531013cf service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:32.506998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a961e07-eb62-4c9c-ad04-dccd531013cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:32.509868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-9a961e07-eb62-4c9c-ad04-dccd531013cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:32.588074 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a961e07-eb62-4c9c-ad04-dccd531013cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:32.589656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a961e07-eb62-4c9c-ad04-dccd531013cf service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:32.601658 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1297/2621] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:32.732605 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8516545-0dda-4bce-9a4e-e8beeedacbee admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:32.734446 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-f8516545-0dda-4bce-9a4e-e8beeedacbee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:32.835904 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8516545-0dda-4bce-9a4e-e8beeedacbee admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:32.842024 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6d515-680a-47dd-bc85-a568124584c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:32.843012 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6d515-680a-47dd-bc85-a568124584c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:32.843374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6d515-680a-47dd-bc85-a568124584c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:32.843750 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6d515-680a-47dd-bc85-a568124584c2 admin admin] PATH_INFO: `/v3/projects/72d3fd1a02e649ce8ee81b91059c11de` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:32.845818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6d515-680a-47dd-bc85-a568124584c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72d3fd1a02e649ce8ee81b91059c11de)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:32.849098 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6d515-680a-47dd-bc85-a568124584c2 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:32.907808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-17c6d515-680a-47dd-bc85-a568124584c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72d3fd1a02e649ce8ee81b91059c11de', 'request_id': 'req-17c6d515-680a-47dd-bc85-a568124584c2'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:32.915667 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-17c6d515-680a-47dd-bc85-a568124584c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72d3fd1a02e649ce8ee81b91059c11de', 'request_id': 'req-17c6d515-680a-47dd-bc85-a568124584c2'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:32.925434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-17c6d515-680a-47dd-bc85-a568124584c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72d3fd1a02e649ce8ee81b91059c11de', 'request_id': 'req-17c6d515-680a-47dd-bc85-a568124584c2'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:32.939126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-17c6d515-680a-47dd-bc85-a568124584c2 admin admin] The token cache is being invalidate because project 72d3fd1a02e649ce8ee81b91059c11de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:12:32.939424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-17c6d515-680a-47dd-bc85-a568124584c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:32.940088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-17c6d515-680a-47dd-bc85-a568124584c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:32.941695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1325/2622] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:12:32 2022] DELETE /identity/v3/projects/72d3fd1a02e649ce8ee81b91059c11de => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:32.950451 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a961e07-eb62-4c9c-ad04-dccd531013cf service nova] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:32.998082 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a961e07-eb62-4c9c-ad04-dccd531013cf service nova] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:33.079519 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46307a79-42a0-470a-91e3-b178606da615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:33.080574 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46307a79-42a0-470a-91e3-b178606da615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:33.080961 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46307a79-42a0-470a-91e3-b178606da615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:33.081309 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46307a79-42a0-470a-91e3-b178606da615 admin admin] PATH_INFO: `/v3/users/bc2376aa8b1f42f987d51fe92a3df439` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:33.082780 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46307a79-42a0-470a-91e3-b178606da615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc2376aa8b1f42f987d51fe92a3df439)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:33.084913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46307a79-42a0-470a-91e3-b178606da615 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:33.154890 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-46307a79-42a0-470a-91e3-b178606da615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc2376aa8b1f42f987d51fe92a3df439'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:33.166802 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-46307a79-42a0-470a-91e3-b178606da615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc2376aa8b1f42f987d51fe92a3df439'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:33.176873 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-46307a79-42a0-470a-91e3-b178606da615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc2376aa8b1f42f987d51fe92a3df439'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:33.177705 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-46307a79-42a0-470a-91e3-b178606da615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc2376aa8b1f42f987d51fe92a3df439'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:33.178503 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-46307a79-42a0-470a-91e3-b178606da615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc2376aa8b1f42f987d51fe92a3df439'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:33.200416 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1298/2623] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:12:32 2022] DELETE /identity/v3/users/bc2376aa8b1f42f987d51fe92a3df439 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:33.516446 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6d515-680a-47dd-bc85-a568124584c2 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:33.518004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-17c6d515-680a-47dd-bc85-a568124584c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:33.537693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6d515-680a-47dd-bc85-a568124584c2 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:33.591308 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:33.610584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2e5e0-eec2-467d-86b0-20ed2318ec48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:33.611564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2e5e0-eec2-467d-86b0-20ed2318ec48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:33.611911 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2e5e0-eec2-467d-86b0-20ed2318ec48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:33.612257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2e5e0-eec2-467d-86b0-20ed2318ec48 admin admin] PATH_INFO: `/v3/projects/f3b9a20ecb9c49a1b0d574daf4861dca` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:33.613611 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2e5e0-eec2-467d-86b0-20ed2318ec48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3b9a20ecb9c49a1b0d574daf4861dca)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:33.615600 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2e5e0-eec2-467d-86b0-20ed2318ec48 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:33.670868 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-e8f2e5e0-eec2-467d-86b0-20ed2318ec48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3b9a20ecb9c49a1b0d574daf4861dca', 'request_id': 'req-e8f2e5e0-eec2-467d-86b0-20ed2318ec48'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:33.676587 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-e8f2e5e0-eec2-467d-86b0-20ed2318ec48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3b9a20ecb9c49a1b0d574daf4861dca', 'request_id': 'req-e8f2e5e0-eec2-467d-86b0-20ed2318ec48'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:33.686656 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-e8f2e5e0-eec2-467d-86b0-20ed2318ec48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3b9a20ecb9c49a1b0d574daf4861dca', 'request_id': 'req-e8f2e5e0-eec2-467d-86b0-20ed2318ec48'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:33.701597 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-e8f2e5e0-eec2-467d-86b0-20ed2318ec48 admin admin] The token cache is being invalidate because project f3b9a20ecb9c49a1b0d574daf4861dca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:12:33.701905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-e8f2e5e0-eec2-467d-86b0-20ed2318ec48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:33.702636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-e8f2e5e0-eec2-467d-86b0-20ed2318ec48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:12:33.708843 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1326/2624] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:12:33 2022] DELETE /identity/v3/projects/f3b9a20ecb9c49a1b0d574daf4861dca => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:33.710392 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d27f2-274f-472a-ab70-be7db99c2e7b None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:33.710717 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d27f2-274f-472a-ab70-be7db99c2e7b None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:33.711008 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d27f2-274f-472a-ab70-be7db99c2e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:33.735163 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-836d27f2-274f-472a-ab70-be7db99c2e7b None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:33.736414 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-836d27f2-274f-472a-ab70-be7db99c2e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:33.741950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:33.807143 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1299/2625] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:33.815782 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1aaa0-64f4-46ab-b764-d62f08c36e8d None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:33.816096 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1aaa0-64f4-46ab-b764-d62f08c36e8d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:33.816379 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1aaa0-64f4-46ab-b764-d62f08c36e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:33.836425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-c3a1aaa0-64f4-46ab-b764-d62f08c36e8d None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:33.837697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c3a1aaa0-64f4-46ab-b764-d62f08c36e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:33.846275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1327/2626] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:33.850702 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d27f2-274f-472a-ab70-be7db99c2e7b None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:33.861787 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d27f2-274f-472a-ab70-be7db99c2e7b None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:33.867536 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01d35d-4a95-4320-acb5-83b02c733a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:33.867904 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01d35d-4a95-4320-acb5-83b02c733a8e admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:33.868193 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01d35d-4a95-4320-acb5-83b02c733a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:33.868472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01d35d-4a95-4320-acb5-83b02c733a8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:33.869273 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01d35d-4a95-4320-acb5-83b02c733a8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:12:33.869553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01d35d-4a95-4320-acb5-83b02c733a8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:33.871169 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01d35d-4a95-4320-acb5-83b02c733a8e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:33.884833 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1300/2627] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:12:33.889403 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1aaa0-64f4-46ab-b764-d62f08c36e8d None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:33.899424 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1aaa0-64f4-46ab-b764-d62f08c36e8d None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:33.906010 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e64fc6-1078-48f9-b7a4-bbf0c11574da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:33.906690 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e64fc6-1078-48f9-b7a4-bbf0c11574da admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:33.907333 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e64fc6-1078-48f9-b7a4-bbf0c11574da admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:33.907460 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e64fc6-1078-48f9-b7a4-bbf0c11574da admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:33.908170 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e64fc6-1078-48f9-b7a4-bbf0c11574da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:33.910384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e64fc6-1078-48f9-b7a4-bbf0c11574da admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:33.940861 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1328/2628] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:12:33 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:33.944829 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01d35d-4a95-4320-acb5-83b02c733a8e admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:33.956112 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01d35d-4a95-4320-acb5-83b02c733a8e admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:34.029428 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438e8de-010f-453e-8e6b-b5922b0c146d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:34.030092 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438e8de-010f-453e-8e6b-b5922b0c146d admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:34.030439 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438e8de-010f-453e-8e6b-b5922b0c146d admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:34.030722 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438e8de-010f-453e-8e6b-b5922b0c146d admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:34.031421 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438e8de-010f-453e-8e6b-b5922b0c146d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:34.033366 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438e8de-010f-453e-8e6b-b5922b0c146d admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:34.055832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1301/2629] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:12:33 2022] POST /identity/v3/users => generated 285 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:12:34.060099 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e64fc6-1078-48f9-b7a4-bbf0c11574da admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:34.071238 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e64fc6-1078-48f9-b7a4-bbf0c11574da admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:34.076136 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd84c0-e648-400f-8a05-cfe13e5ad8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:34.076978 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd84c0-e648-400f-8a05-cfe13e5ad8ee admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:34.077335 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd84c0-e648-400f-8a05-cfe13e5ad8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:34.077601 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd84c0-e648-400f-8a05-cfe13e5ad8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:34.078341 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd84c0-e648-400f-8a05-cfe13e5ad8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:34.079947 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd84c0-e648-400f-8a05-cfe13e5ad8ee admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:34.091754 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1329/2630] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:34.095816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438e8de-010f-453e-8e6b-b5922b0c146d admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:34.107286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438e8de-010f-453e-8e6b-b5922b0c146d admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:34.112206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f828b-ff6a-4e04-8305-dd4d54a5426f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:34.112931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f828b-ff6a-4e04-8305-dd4d54a5426f admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:34.113218 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f828b-ff6a-4e04-8305-dd4d54a5426f admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:34.113497 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f828b-ff6a-4e04-8305-dd4d54a5426f admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:34.114230 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f828b-ff6a-4e04-8305-dd4d54a5426f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:34.115849 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f828b-ff6a-4e04-8305-dd4d54a5426f admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:34.130751 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1302/2631] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:12:34.133715 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd84c0-e648-400f-8a05-cfe13e5ad8ee admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:34.146943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd84c0-e648-400f-8a05-cfe13e5ad8ee admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:34.151778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13f0d2-ba82-4784-a40f-3a29f5c9d9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:34.153225 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13f0d2-ba82-4784-a40f-3a29f5c9d9e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:34.153553 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13f0d2-ba82-4784-a40f-3a29f5c9d9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:34.153784 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13f0d2-ba82-4784-a40f-3a29f5c9d9e5 admin admin] PATH_INFO: `/v3/projects/d314534667384d8ea351d2180a35c552/users/2d197897b53f4ad0a9b4e7413c986729/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:34.173947 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab13f0d2-ba82-4784-a40f-3a29f5c9d9e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d314534667384d8ea351d2180a35c552, user_id=2d197897b53f4ad0a9b4e7413c986729, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:34.177000 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab13f0d2-ba82-4784-a40f-3a29f5c9d9e5 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:34.201246 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1330/2632] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:12:34 2022] PUT /identity/v3/projects/d314534667384d8ea351d2180a35c552/users/2d197897b53f4ad0a9b4e7413c986729/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:12:34.212200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add87daf-a6f3-4d9a-8373-946ff5221709 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:34.212529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add87daf-a6f3-4d9a-8373-946ff5221709 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:34.212922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add87daf-a6f3-4d9a-8373-946ff5221709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:34.255879 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-add87daf-a6f3-4d9a-8373-946ff5221709 None None] MFA Rules not processed for user `2d197897b53f4ad0a9b4e7413c986729`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:34.257319 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-add87daf-a6f3-4d9a-8373-946ff5221709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:34.343935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1303/2633] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:12:34.358477 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13f0d2-ba82-4784-a40f-3a29f5c9d9e5 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:34.361020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ab13f0d2-ba82-4784-a40f-3a29f5c9d9e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:34.380979 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef883c-6553-47c7-b266-83426cca2f36 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:34.381285 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef883c-6553-47c7-b266-83426cca2f36 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:34.381548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef883c-6553-47c7-b266-83426cca2f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:34.390713 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:34.414026 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-5fef883c-6553-47c7-b266-83426cca2f36 None None] MFA Rules not processed for user `2d0d387beaf842c3b5978210e504fbed`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:12:34.415314 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-5fef883c-6553-47c7-b266-83426cca2f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:34.495515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13f0d2-ba82-4784-a40f-3a29f5c9d9e5 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:34.501576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784a02f-bbfe-41e3-b293-060933e344fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5c9ed6f9fd542069cf914003b636c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:34.503063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784a02f-bbfe-41e3-b293-060933e344fe service glance] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:34.503556 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784a02f-bbfe-41e3-b293-060933e344fe service glance] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:34.504034 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784a02f-bbfe-41e3-b293-060933e344fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:34.518169 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784a02f-bbfe-41e3-b293-060933e344fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:34.520037 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784a02f-bbfe-41e3-b293-060933e344fe service glance] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:34.524246 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1304/2634] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:12:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:12:34.531618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1331/2635] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:34 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:34.888865 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef883c-6553-47c7-b266-83426cca2f36 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:34.891073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-5fef883c-6553-47c7-b266-83426cca2f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:34.966718 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef883c-6553-47c7-b266-83426cca2f36 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:34.972391 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365abe9-0a77-4f08-8e2a-6dbc7d737672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:34.973374 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365abe9-0a77-4f08-8e2a-6dbc7d737672 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:34.973806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365abe9-0a77-4f08-8e2a-6dbc7d737672 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:34.974220 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365abe9-0a77-4f08-8e2a-6dbc7d737672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:34.976343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-9365abe9-0a77-4f08-8e2a-6dbc7d737672 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:34.988514 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365abe9-0a77-4f08-8e2a-6dbc7d737672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:34.990272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365abe9-0a77-4f08-8e2a-6dbc7d737672 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:35.000962 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1305/2636] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:44.526025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:44.995816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=49 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:12:48.273742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784a02f-bbfe-41e3-b293-060933e344fe service glance] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:12:48.275253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-c784a02f-bbfe-41e3-b293-060933e344fe service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:48.353936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784a02f-bbfe-41e3-b293-060933e344fe service glance] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:12:48.360078 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17b628-923d-4fc8-879d-9ac549b3c991 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:12:48.360763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17b628-923d-4fc8-879d-9ac549b3c991 service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:12:48.361056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17b628-923d-4fc8-879d-9ac549b3c991 service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:12:48.361338 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17b628-923d-4fc8-879d-9ac549b3c991 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:12:48.363149 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-4e17b628-923d-4fc8-879d-9ac549b3c991 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:12:48.376267 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e17b628-923d-4fc8-879d-9ac549b3c991 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:12:48.377849 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e17b628-923d-4fc8-879d-9ac549b3c991 service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:12:48.389110 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1332/2637] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:12:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:12:58.384277 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=34 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:13:05.787810 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365abe9-0a77-4f08-8e2a-6dbc7d737672 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:05.852207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365abe9-0a77-4f08-8e2a-6dbc7d737672 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:05.856854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee8653-d539-4498-a29f-94464b7698ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:05.857804 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee8653-d539-4498-a29f-94464b7698ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:05.858088 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee8653-d539-4498-a29f-94464b7698ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:05.858478 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee8653-d539-4498-a29f-94464b7698ec admin admin] PATH_INFO: `/v3/users/2d197897b53f4ad0a9b4e7413c986729` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:05.859719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ee8653-d539-4498-a29f-94464b7698ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d197897b53f4ad0a9b4e7413c986729)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:05.861687 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ee8653-d539-4498-a29f-94464b7698ec admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:05.910716 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a0ee8653-d539-4498-a29f-94464b7698ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d197897b53f4ad0a9b4e7413c986729'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:05.917519 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a0ee8653-d539-4498-a29f-94464b7698ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d197897b53f4ad0a9b4e7413c986729'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:05.924789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a0ee8653-d539-4498-a29f-94464b7698ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d197897b53f4ad0a9b4e7413c986729'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:05.925407 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a0ee8653-d539-4498-a29f-94464b7698ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d197897b53f4ad0a9b4e7413c986729'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:05.926011 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a0ee8653-d539-4498-a29f-94464b7698ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d197897b53f4ad0a9b4e7413c986729'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:05.944037 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1306/2638] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:13:05 2022] DELETE /identity/v3/users/2d197897b53f4ad0a9b4e7413c986729 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:13:05.948661 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639a61e-a596-44fa-a278-9969283f74f4 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:05.949196 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639a61e-a596-44fa-a278-9969283f74f4 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:05.949381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639a61e-a596-44fa-a278-9969283f74f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:05.968432 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-4639a61e-a596-44fa-a278-9969283f74f4 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:13:05.969803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-4639a61e-a596-44fa-a278-9969283f74f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:06.030423 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1333/2639] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:13:06.040158 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee8653-d539-4498-a29f-94464b7698ec admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:06.041629 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a0ee8653-d539-4498-a29f-94464b7698ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:06.099920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee8653-d539-4498-a29f-94464b7698ec admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:06.103677 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f3e8c-5161-4c6d-af1b-7fc77f6c16b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:06.105323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f3e8c-5161-4c6d-af1b-7fc77f6c16b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:06.105614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f3e8c-5161-4c6d-af1b-7fc77f6c16b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:06.105895 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f3e8c-5161-4c6d-af1b-7fc77f6c16b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:06.115340 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f3e8c-5161-4c6d-af1b-7fc77f6c16b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:06.116819 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f3e8c-5161-4c6d-af1b-7fc77f6c16b4 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:06.127448 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1307/2640] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:13:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:13:06.405948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639a61e-a596-44fa-a278-9969283f74f4 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:06.407539 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-4639a61e-a596-44fa-a278-9969283f74f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:06.420035 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639a61e-a596-44fa-a278-9969283f74f4 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:06.425606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e99f6a-cabe-4d9c-9518-c0339db3b25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:06.426496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e99f6a-cabe-4d9c-9518-c0339db3b25c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:06.426779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e99f6a-cabe-4d9c-9518-c0339db3b25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:06.427054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e99f6a-cabe-4d9c-9518-c0339db3b25c admin admin] PATH_INFO: `/v3/projects/d314534667384d8ea351d2180a35c552` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:06.428234 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e99f6a-cabe-4d9c-9518-c0339db3b25c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d314534667384d8ea351d2180a35c552)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:06.430190 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e99f6a-cabe-4d9c-9518-c0339db3b25c admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:06.478348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-74e99f6a-cabe-4d9c-9518-c0339db3b25c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd314534667384d8ea351d2180a35c552', 'request_id': 'req-74e99f6a-cabe-4d9c-9518-c0339db3b25c'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:06.484339 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-74e99f6a-cabe-4d9c-9518-c0339db3b25c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd314534667384d8ea351d2180a35c552', 'request_id': 'req-74e99f6a-cabe-4d9c-9518-c0339db3b25c'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:06.492097 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-74e99f6a-cabe-4d9c-9518-c0339db3b25c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd314534667384d8ea351d2180a35c552', 'request_id': 'req-74e99f6a-cabe-4d9c-9518-c0339db3b25c'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:06.505041 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-74e99f6a-cabe-4d9c-9518-c0339db3b25c admin admin] The token cache is being invalidate because project d314534667384d8ea351d2180a35c552 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:13:06.505329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-74e99f6a-cabe-4d9c-9518-c0339db3b25c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:06.505975 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-74e99f6a-cabe-4d9c-9518-c0339db3b25c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:06.507531 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1334/2641] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:13:06 2022] DELETE /identity/v3/projects/d314534667384d8ea351d2180a35c552 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:13:06.512413 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74b00b-1d99-4daf-a23f-bd926a0f2385 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:06.512721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74b00b-1d99-4daf-a23f-bd926a0f2385 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:06.513020 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74b00b-1d99-4daf-a23f-bd926a0f2385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:06.531272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-6d74b00b-1d99-4daf-a23f-bd926a0f2385 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:13:06.532464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6d74b00b-1d99-4daf-a23f-bd926a0f2385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:06.550499 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:13:06.594767 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1308/2642] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:13:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:13:06.602732 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbb995-5579-4ed5-9d0e-861053cf23b3 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:06.603025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbb995-5579-4ed5-9d0e-861053cf23b3 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:06.603305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbb995-5579-4ed5-9d0e-861053cf23b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:06.621986 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-e7fbb995-5579-4ed5-9d0e-861053cf23b3 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:13:06.623930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e7fbb995-5579-4ed5-9d0e-861053cf23b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:06.632068 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1335/2643] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:13:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:13:06.636407 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74b00b-1d99-4daf-a23f-bd926a0f2385 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:06.646509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74b00b-1d99-4daf-a23f-bd926a0f2385 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:06.651446 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b5abd-a15f-4a76-bc83-477d4ff41ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:06.652129 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b5abd-a15f-4a76-bc83-477d4ff41ebc admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:06.652408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b5abd-a15f-4a76-bc83-477d4ff41ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:06.652684 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b5abd-a15f-4a76-bc83-477d4ff41ebc admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:06.653450 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b5abd-a15f-4a76-bc83-477d4ff41ebc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:13:06.653758 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b5abd-a15f-4a76-bc83-477d4ff41ebc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:06.655470 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b5abd-a15f-4a76-bc83-477d4ff41ebc admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:06.669236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1309/2644] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:13:06.674630 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbb995-5579-4ed5-9d0e-861053cf23b3 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:06.683955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbb995-5579-4ed5-9d0e-861053cf23b3 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:06.689742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5794a4-e6a1-468c-8307-701e66f1cb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:06.690388 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5794a4-e6a1-468c-8307-701e66f1cb46 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:06.690682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5794a4-e6a1-468c-8307-701e66f1cb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:06.690962 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5794a4-e6a1-468c-8307-701e66f1cb46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:06.691660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5794a4-e6a1-468c-8307-701e66f1cb46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:06.693859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5794a4-e6a1-468c-8307-701e66f1cb46 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:06.719390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1336/2645] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:13:06 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:13:06.723659 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b5abd-a15f-4a76-bc83-477d4ff41ebc admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:06.734951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b5abd-a15f-4a76-bc83-477d4ff41ebc admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:06.794685 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed314e8b-a4f9-4b89-bf12-4c36aabe4e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:06.796197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed314e8b-a4f9-4b89-bf12-4c36aabe4e2a admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:06.796484 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed314e8b-a4f9-4b89-bf12-4c36aabe4e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:06.796794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed314e8b-a4f9-4b89-bf12-4c36aabe4e2a admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:06.797495 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed314e8b-a4f9-4b89-bf12-4c36aabe4e2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:06.799577 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed314e8b-a4f9-4b89-bf12-4c36aabe4e2a admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:06.820405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1310/2646] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:13:06 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:13:06.824647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5794a4-e6a1-468c-8307-701e66f1cb46 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:06.835295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5794a4-e6a1-468c-8307-701e66f1cb46 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:06.839949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0567b71-cd92-45b3-862d-f2f1067b2045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:06.840602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0567b71-cd92-45b3-862d-f2f1067b2045 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:06.840924 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0567b71-cd92-45b3-862d-f2f1067b2045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:06.841208 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0567b71-cd92-45b3-862d-f2f1067b2045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:06.842440 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0567b71-cd92-45b3-862d-f2f1067b2045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:06.844079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0567b71-cd92-45b3-862d-f2f1067b2045 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:06.855159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1337/2647] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:13:06.859893 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed314e8b-a4f9-4b89-bf12-4c36aabe4e2a admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:06.870050 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed314e8b-a4f9-4b89-bf12-4c36aabe4e2a admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:06.877073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ec7df-4c47-4d25-bd44-8215dd676830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:06.877870 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ec7df-4c47-4d25-bd44-8215dd676830 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:06.878261 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ec7df-4c47-4d25-bd44-8215dd676830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:06.878655 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ec7df-4c47-4d25-bd44-8215dd676830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:06.879496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ec7df-4c47-4d25-bd44-8215dd676830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:06.881350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ec7df-4c47-4d25-bd44-8215dd676830 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:06.892282 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1311/2648] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:13:06.896479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0567b71-cd92-45b3-862d-f2f1067b2045 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:06.907269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0567b71-cd92-45b3-862d-f2f1067b2045 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:06.911882 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eeacbd-7705-42ba-a9ec-494449558702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:06.912793 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eeacbd-7705-42ba-a9ec-494449558702 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:06.913077 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eeacbd-7705-42ba-a9ec-494449558702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:06.913353 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eeacbd-7705-42ba-a9ec-494449558702 admin admin] PATH_INFO: `/v3/projects/d117938a18a34ad0a49aac69e6bf2c45/users/8feb2986b9a04898aa958e0c9d8380c3/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:06.928454 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eeacbd-7705-42ba-a9ec-494449558702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d117938a18a34ad0a49aac69e6bf2c45, user_id=8feb2986b9a04898aa958e0c9d8380c3, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:06.931123 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eeacbd-7705-42ba-a9ec-494449558702 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:06.944377 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1338/2649] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:13:06 2022] PUT /identity/v3/projects/d117938a18a34ad0a49aac69e6bf2c45/users/8feb2986b9a04898aa958e0c9d8380c3/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:13:06.956226 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc432dd-d384-42f6-93e6-a2988e76a387 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:06.956513 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc432dd-d384-42f6-93e6-a2988e76a387 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:06.956830 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc432dd-d384-42f6-93e6-a2988e76a387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:06.985084 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-cbc432dd-d384-42f6-93e6-a2988e76a387 None None] MFA Rules not processed for user `8feb2986b9a04898aa958e0c9d8380c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:13:06.986303 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-cbc432dd-d384-42f6-93e6-a2988e76a387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:07.049528 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1312/2650] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:13:06 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:13:07.060797 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eeacbd-7705-42ba-a9ec-494449558702 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:07.062686 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a6eeacbd-7705-42ba-a9ec-494449558702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:07.084981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:13:07.122154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eeacbd-7705-42ba-a9ec-494449558702 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:07.125886 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719cf007-8a9b-4215-b803-7f610885a52e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:07.126523 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719cf007-8a9b-4215-b803-7f610885a52e service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:07.126805 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719cf007-8a9b-4215-b803-7f610885a52e service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:07.127078 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719cf007-8a9b-4215-b803-7f610885a52e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:07.135243 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719cf007-8a9b-4215-b803-7f610885a52e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:07.136655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719cf007-8a9b-4215-b803-7f610885a52e service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:07.145427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1339/2651] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:13:07 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:13:07.170868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc432dd-d384-42f6-93e6-a2988e76a387 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:07.190373 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:13:07.235660 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc432dd-d384-42f6-93e6-a2988e76a387 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:07.240306 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c1c08-f155-4c58-a340-7a7c936b62cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:07.241198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c1c08-f155-4c58-a340-7a7c936b62cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:07.241480 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c1c08-f155-4c58-a340-7a7c936b62cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:07.241753 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c1c08-f155-4c58-a340-7a7c936b62cf admin admin] PATH_INFO: `/v3/users/8feb2986b9a04898aa958e0c9d8380c3` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:07.242936 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c1c08-f155-4c58-a340-7a7c936b62cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8feb2986b9a04898aa958e0c9d8380c3)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:07.244854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c1c08-f155-4c58-a340-7a7c936b62cf admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:07.291969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8a6c1c08-f155-4c58-a340-7a7c936b62cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8feb2986b9a04898aa958e0c9d8380c3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:07.299202 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8a6c1c08-f155-4c58-a340-7a7c936b62cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8feb2986b9a04898aa958e0c9d8380c3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:07.306765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8a6c1c08-f155-4c58-a340-7a7c936b62cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8feb2986b9a04898aa958e0c9d8380c3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:07.307402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8a6c1c08-f155-4c58-a340-7a7c936b62cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8feb2986b9a04898aa958e0c9d8380c3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:07.308021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8a6c1c08-f155-4c58-a340-7a7c936b62cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8feb2986b9a04898aa958e0c9d8380c3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:07.325222 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1313/2652] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:13:07 2022] DELETE /identity/v3/users/8feb2986b9a04898aa958e0c9d8380c3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:13:07.330415 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6b053-74d5-451a-a350-a7da4166cb1c None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:07.330713 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6b053-74d5-451a-a350-a7da4166cb1c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:07.330999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6b053-74d5-451a-a350-a7da4166cb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:07.348335 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-0af6b053-74d5-451a-a350-a7da4166cb1c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:13:07.349537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0af6b053-74d5-451a-a350-a7da4166cb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:07.408334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1340/2653] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:13:07.418199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c1c08-f155-4c58-a340-7a7c936b62cf admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:07.419625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8a6c1c08-f155-4c58-a340-7a7c936b62cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:07.476884 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c1c08-f155-4c58-a340-7a7c936b62cf admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:07.480506 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f860ab-5bd7-483f-bacb-46d2efbe7826 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:07.481181 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f860ab-5bd7-483f-bacb-46d2efbe7826 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:07.481468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f860ab-5bd7-483f-bacb-46d2efbe7826 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:07.481745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f860ab-5bd7-483f-bacb-46d2efbe7826 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:07.491577 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f860ab-5bd7-483f-bacb-46d2efbe7826 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:07.493078 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f860ab-5bd7-483f-bacb-46d2efbe7826 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:07.503219 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1314/2654] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:13:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:13:07.769707 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6b053-74d5-451a-a350-a7da4166cb1c None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:07.771255 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0af6b053-74d5-451a-a350-a7da4166cb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:07.784460 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6b053-74d5-451a-a350-a7da4166cb1c None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:07.789138 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e219cde-7364-435d-bf6b-cb99653ce5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:07.790014 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e219cde-7364-435d-bf6b-cb99653ce5e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:07.790293 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e219cde-7364-435d-bf6b-cb99653ce5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:07.790602 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e219cde-7364-435d-bf6b-cb99653ce5e3 admin admin] PATH_INFO: `/v3/projects/d117938a18a34ad0a49aac69e6bf2c45` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:07.791801 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e219cde-7364-435d-bf6b-cb99653ce5e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d117938a18a34ad0a49aac69e6bf2c45)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:07.793801 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e219cde-7364-435d-bf6b-cb99653ce5e3 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:07.842932 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-7e219cde-7364-435d-bf6b-cb99653ce5e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd117938a18a34ad0a49aac69e6bf2c45', 'request_id': 'req-7e219cde-7364-435d-bf6b-cb99653ce5e3'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:07.848450 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-7e219cde-7364-435d-bf6b-cb99653ce5e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd117938a18a34ad0a49aac69e6bf2c45', 'request_id': 'req-7e219cde-7364-435d-bf6b-cb99653ce5e3'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:07.856167 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-7e219cde-7364-435d-bf6b-cb99653ce5e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd117938a18a34ad0a49aac69e6bf2c45', 'request_id': 'req-7e219cde-7364-435d-bf6b-cb99653ce5e3'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:07.868205 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-7e219cde-7364-435d-bf6b-cb99653ce5e3 admin admin] The token cache is being invalidate because project d117938a18a34ad0a49aac69e6bf2c45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:13:07.868497 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-7e219cde-7364-435d-bf6b-cb99653ce5e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:07.869172 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-7e219cde-7364-435d-bf6b-cb99653ce5e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:07.870732 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1341/2655] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:13:07 2022] DELETE /identity/v3/projects/d117938a18a34ad0a49aac69e6bf2c45 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:13:07.876880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740fc05-eb2c-456b-b744-de6ce30346ff None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:07.877427 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740fc05-eb2c-456b-b744-de6ce30346ff None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:07.877534 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740fc05-eb2c-456b-b744-de6ce30346ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:07.901211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-f740fc05-eb2c-456b-b744-de6ce30346ff None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:13:07.902359 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-f740fc05-eb2c-456b-b744-de6ce30346ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:07.937998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:13:07.969073 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1315/2656] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:13:07.977243 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa7afe-d926-4a76-93a3-b1c907f0ca9f None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:07.977567 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa7afe-d926-4a76-93a3-b1c907f0ca9f None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:07.977852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa7afe-d926-4a76-93a3-b1c907f0ca9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:07.997542 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-8cfa7afe-d926-4a76-93a3-b1c907f0ca9f None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:13:07.999382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8cfa7afe-d926-4a76-93a3-b1c907f0ca9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:08.007710 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1342/2657] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:13:08.012056 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740fc05-eb2c-456b-b744-de6ce30346ff None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:08.021980 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740fc05-eb2c-456b-b744-de6ce30346ff None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:08.026614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99726452-2a98-4cf1-b06c-8b89bc1e375f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:08.027266 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99726452-2a98-4cf1-b06c-8b89bc1e375f admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:08.027552 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99726452-2a98-4cf1-b06c-8b89bc1e375f admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:08.027832 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99726452-2a98-4cf1-b06c-8b89bc1e375f admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:08.028582 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99726452-2a98-4cf1-b06c-8b89bc1e375f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:13:08.028922 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99726452-2a98-4cf1-b06c-8b89bc1e375f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:08.030559 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99726452-2a98-4cf1-b06c-8b89bc1e375f admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:08.042097 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1316/2658] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:13:08.046917 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa7afe-d926-4a76-93a3-b1c907f0ca9f None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:08.056892 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa7afe-d926-4a76-93a3-b1c907f0ca9f None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:08.062437 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82349d8e-29b6-421e-ad44-eea885b58eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:08.063134 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82349d8e-29b6-421e-ad44-eea885b58eac admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:08.063422 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82349d8e-29b6-421e-ad44-eea885b58eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:08.063704 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82349d8e-29b6-421e-ad44-eea885b58eac admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:08.064398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82349d8e-29b6-421e-ad44-eea885b58eac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:08.066411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82349d8e-29b6-421e-ad44-eea885b58eac admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:08.097125 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1343/2659] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:13:08 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:13:08.101529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99726452-2a98-4cf1-b06c-8b89bc1e375f admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:08.112164 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99726452-2a98-4cf1-b06c-8b89bc1e375f admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:08.167848 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88a5a8-51d2-4f21-90f2-3afccbf5f89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:08.168642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88a5a8-51d2-4f21-90f2-3afccbf5f89f admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:08.169026 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88a5a8-51d2-4f21-90f2-3afccbf5f89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:08.169343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88a5a8-51d2-4f21-90f2-3afccbf5f89f admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:08.170592 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88a5a8-51d2-4f21-90f2-3afccbf5f89f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:08.172521 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88a5a8-51d2-4f21-90f2-3afccbf5f89f admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:08.192625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1317/2660] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:13:08 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:13:08.197258 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82349d8e-29b6-421e-ad44-eea885b58eac admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:08.208112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82349d8e-29b6-421e-ad44-eea885b58eac admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:08.212758 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb127e-52e7-46ca-bd92-62bbf147177e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:08.213421 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb127e-52e7-46ca-bd92-62bbf147177e admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:08.213709 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb127e-52e7-46ca-bd92-62bbf147177e admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:08.213991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb127e-52e7-46ca-bd92-62bbf147177e admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:08.214732 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb127e-52e7-46ca-bd92-62bbf147177e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:08.216345 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb127e-52e7-46ca-bd92-62bbf147177e admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:08.227231 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1344/2661] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:13:08.231525 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88a5a8-51d2-4f21-90f2-3afccbf5f89f admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:08.242255 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88a5a8-51d2-4f21-90f2-3afccbf5f89f admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:08.246847 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0a68b-3f14-4655-8eae-bf7587b58580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:08.247502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0a68b-3f14-4655-8eae-bf7587b58580 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:08.247790 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0a68b-3f14-4655-8eae-bf7587b58580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:08.248065 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0a68b-3f14-4655-8eae-bf7587b58580 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:08.249422 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0a68b-3f14-4655-8eae-bf7587b58580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:08.251007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0a68b-3f14-4655-8eae-bf7587b58580 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:08.261007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1318/2662] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:13:08.265243 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb127e-52e7-46ca-bd92-62bbf147177e admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:08.275266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb127e-52e7-46ca-bd92-62bbf147177e admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:08.280863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9a715-f4a9-49b5-8598-bf81e4580398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:08.281746 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9a715-f4a9-49b5-8598-bf81e4580398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:08.282035 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9a715-f4a9-49b5-8598-bf81e4580398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:08.282311 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9a715-f4a9-49b5-8598-bf81e4580398 admin admin] PATH_INFO: `/v3/projects/41258d63fe9d4d1ea971b99420cb8b5e/users/ec7d47d6a5444ef6860e219e48f086b5/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:08.284618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:13:08.311932 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d9a715-f4a9-49b5-8598-bf81e4580398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41258d63fe9d4d1ea971b99420cb8b5e, user_id=ec7d47d6a5444ef6860e219e48f086b5, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:08.317735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d9a715-f4a9-49b5-8598-bf81e4580398 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:08.331818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1345/2663] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:13:08 2022] PUT /identity/v3/projects/41258d63fe9d4d1ea971b99420cb8b5e/users/ec7d47d6a5444ef6860e219e48f086b5/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:13:08.342690 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50391cf2-9c70-4019-ac34-4027ab228026 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:08.342982 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50391cf2-9c70-4019-ac34-4027ab228026 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:08.343264 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50391cf2-9c70-4019-ac34-4027ab228026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:08.369878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-50391cf2-9c70-4019-ac34-4027ab228026 None None] MFA Rules not processed for user `ec7d47d6a5444ef6860e219e48f086b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:13:08.371093 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-50391cf2-9c70-4019-ac34-4027ab228026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:08.434956 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1319/2664] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:13:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:13:08.445958 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9a715-f4a9-49b5-8598-bf81e4580398 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:08.447775 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-33d9a715-f4a9-49b5-8598-bf81e4580398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:08.505064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9a715-f4a9-49b5-8598-bf81e4580398 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:08.508803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ebcfe-42d4-46f1-8adb-59c768262b88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:08.509448 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ebcfe-42d4-46f1-8adb-59c768262b88 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:08.509738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ebcfe-42d4-46f1-8adb-59c768262b88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:08.510024 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ebcfe-42d4-46f1-8adb-59c768262b88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:08.518468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ebcfe-42d4-46f1-8adb-59c768262b88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:08.519907 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ebcfe-42d4-46f1-8adb-59c768262b88 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:08.528979 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1346/2665] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:13:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:13:18.431801 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:13:18.524691 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:13:22.282913 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe459d-1e57-4933-a718-ab7329433bbc None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:22.283468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe459d-1e57-4933-a718-ab7329433bbc None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:22.283468 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe459d-1e57-4933-a718-ab7329433bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:22.302709 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-d7fe459d-1e57-4933-a718-ab7329433bbc None None] MFA Rules not processed for user `24a41c09f2e4487f9897c9888a0d752f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:13:22.303947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d7fe459d-1e57-4933-a718-ab7329433bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:22.365338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1320/2666] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:13:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:13:25.813722 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ebcfe-42d4-46f1-8adb-59c768262b88 service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:25.896119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ebcfe-42d4-46f1-8adb-59c768262b88 service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:25.900724 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de527f69-3c0a-4530-9397-7e376d047141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:25.901699 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de527f69-3c0a-4530-9397-7e376d047141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:25.902053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de527f69-3c0a-4530-9397-7e376d047141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:25.902411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de527f69-3c0a-4530-9397-7e376d047141 admin admin] PATH_INFO: `/v3/users/ec7d47d6a5444ef6860e219e48f086b5` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:25.903820 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de527f69-3c0a-4530-9397-7e376d047141 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec7d47d6a5444ef6860e219e48f086b5)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:25.905903 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de527f69-3c0a-4530-9397-7e376d047141 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:25.964004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-de527f69-3c0a-4530-9397-7e376d047141 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec7d47d6a5444ef6860e219e48f086b5'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:25.978592 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-de527f69-3c0a-4530-9397-7e376d047141 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec7d47d6a5444ef6860e219e48f086b5'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:25.986374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-de527f69-3c0a-4530-9397-7e376d047141 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec7d47d6a5444ef6860e219e48f086b5'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:25.994236 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-de527f69-3c0a-4530-9397-7e376d047141 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec7d47d6a5444ef6860e219e48f086b5'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:25.994885 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-de527f69-3c0a-4530-9397-7e376d047141 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec7d47d6a5444ef6860e219e48f086b5'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:26.001301 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1347/2667] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:13:25 2022] DELETE /identity/v3/users/ec7d47d6a5444ef6860e219e48f086b5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:13:26.005928 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378302b4-4495-44c0-8353-7a7a6618427c None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:26.006421 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378302b4-4495-44c0-8353-7a7a6618427c None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:26.006797 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378302b4-4495-44c0-8353-7a7a6618427c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:26.025135 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-378302b4-4495-44c0-8353-7a7a6618427c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:13:26.026362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-378302b4-4495-44c0-8353-7a7a6618427c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:26.089288 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1321/2668] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:13:26.099317 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de527f69-3c0a-4530-9397-7e376d047141 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:26.100910 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-de527f69-3c0a-4530-9397-7e376d047141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:26.162181 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de527f69-3c0a-4530-9397-7e376d047141 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:26.166245 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530d58e-7106-4924-9021-6053332139f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:26.166954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530d58e-7106-4924-9021-6053332139f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:26.167257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530d58e-7106-4924-9021-6053332139f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:26.167514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530d58e-7106-4924-9021-6053332139f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:26.178352 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530d58e-7106-4924-9021-6053332139f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:26.179741 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530d58e-7106-4924-9021-6053332139f1 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:26.189938 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1348/2669] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:13:26.470149 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378302b4-4495-44c0-8353-7a7a6618427c None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:26.471605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-378302b4-4495-44c0-8353-7a7a6618427c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:26.484055 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378302b4-4495-44c0-8353-7a7a6618427c None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:26.490395 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dae309-133a-4b9c-a553-b227a4fd5d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:26.491275 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dae309-133a-4b9c-a553-b227a4fd5d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:26.491562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dae309-133a-4b9c-a553-b227a4fd5d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:26.491838 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dae309-133a-4b9c-a553-b227a4fd5d76 admin admin] PATH_INFO: `/v3/projects/41258d63fe9d4d1ea971b99420cb8b5e` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:26.493065 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dae309-133a-4b9c-a553-b227a4fd5d76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41258d63fe9d4d1ea971b99420cb8b5e)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:26.494984 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dae309-133a-4b9c-a553-b227a4fd5d76 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:26.545539 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-51dae309-133a-4b9c-a553-b227a4fd5d76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41258d63fe9d4d1ea971b99420cb8b5e', 'request_id': 'req-51dae309-133a-4b9c-a553-b227a4fd5d76'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:26.551855 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-51dae309-133a-4b9c-a553-b227a4fd5d76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41258d63fe9d4d1ea971b99420cb8b5e', 'request_id': 'req-51dae309-133a-4b9c-a553-b227a4fd5d76'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:26.561237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-51dae309-133a-4b9c-a553-b227a4fd5d76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41258d63fe9d4d1ea971b99420cb8b5e', 'request_id': 'req-51dae309-133a-4b9c-a553-b227a4fd5d76'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:26.572894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-51dae309-133a-4b9c-a553-b227a4fd5d76 admin admin] The token cache is being invalidate because project 41258d63fe9d4d1ea971b99420cb8b5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:13:26.573190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-51dae309-133a-4b9c-a553-b227a4fd5d76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:26.573844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-51dae309-133a-4b9c-a553-b227a4fd5d76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:13:26.575404 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1322/2670] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:13:26 2022] DELETE /identity/v3/projects/41258d63fe9d4d1ea971b99420cb8b5e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:13:26.581003 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b986598-f073-4d45-b9fe-46ca46c0c1c8 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:26.581373 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b986598-f073-4d45-b9fe-46ca46c0c1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:26.581724 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b986598-f073-4d45-b9fe-46ca46c0c1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:26.599732 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-5b986598-f073-4d45-b9fe-46ca46c0c1c8 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:13:26.601125 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5b986598-f073-4d45-b9fe-46ca46c0c1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:26.618695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:13:26.666785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1349/2671] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:13:26.732622 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9a714-0292-48f7-a7aa-06a29d9574a6 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:26.733113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9a714-0292-48f7-a7aa-06a29d9574a6 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:26.733363 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9a714-0292-48f7-a7aa-06a29d9574a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:26.758706 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-2ee9a714-0292-48f7-a7aa-06a29d9574a6 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:13:26.760502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-2ee9a714-0292-48f7-a7aa-06a29d9574a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:26.771009 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1323/2672] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:13:26.775334 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b986598-f073-4d45-b9fe-46ca46c0c1c8 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:26.786088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b986598-f073-4d45-b9fe-46ca46c0c1c8 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:26.791098 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2dee0c-9e27-451f-b9de-1bd11fd502dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:26.791751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2dee0c-9e27-451f-b9de-1bd11fd502dc admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:26.792035 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2dee0c-9e27-451f-b9de-1bd11fd502dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:26.792310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2dee0c-9e27-451f-b9de-1bd11fd502dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:26.793076 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2dee0c-9e27-451f-b9de-1bd11fd502dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:13:26.793384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2dee0c-9e27-451f-b9de-1bd11fd502dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:26.795601 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2dee0c-9e27-451f-b9de-1bd11fd502dc admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:26.807574 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1350/2673] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 22:13:26.812563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9a714-0292-48f7-a7aa-06a29d9574a6 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:26.822175 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9a714-0292-48f7-a7aa-06a29d9574a6 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:26.827024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ccfa4-5dc2-4824-afd5-e2d9d9430b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:26.827680 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ccfa4-5dc2-4824-afd5-e2d9d9430b89 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:26.827963 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ccfa4-5dc2-4824-afd5-e2d9d9430b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:26.828241 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ccfa4-5dc2-4824-afd5-e2d9d9430b89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:26.828979 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ccfa4-5dc2-4824-afd5-e2d9d9430b89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:26.830892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ccfa4-5dc2-4824-afd5-e2d9d9430b89 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:26.857679 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1324/2674] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:13:26 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:13:26.860388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:13:26.861799 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2dee0c-9e27-451f-b9de-1bd11fd502dc admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:26.874772 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2dee0c-9e27-451f-b9de-1bd11fd502dc admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:26.933327 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb965b7-3b94-42d0-bddc-e5cd56ffe5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:26.933998 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb965b7-3b94-42d0-bddc-e5cd56ffe5dd admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:26.934279 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb965b7-3b94-42d0-bddc-e5cd56ffe5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:26.934554 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb965b7-3b94-42d0-bddc-e5cd56ffe5dd admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:26.935995 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb965b7-3b94-42d0-bddc-e5cd56ffe5dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:26.938200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb965b7-3b94-42d0-bddc-e5cd56ffe5dd admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:26.961706 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1351/2675] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:13:26 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:13:26.965594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ccfa4-5dc2-4824-afd5-e2d9d9430b89 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:26.976797 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ccfa4-5dc2-4824-afd5-e2d9d9430b89 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:26.981639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbddcee-10b1-4a03-a10c-2ccf35534802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:26.982880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbddcee-10b1-4a03-a10c-2ccf35534802 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:26.983163 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbddcee-10b1-4a03-a10c-2ccf35534802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:26.983440 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbddcee-10b1-4a03-a10c-2ccf35534802 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:26.984172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbddcee-10b1-4a03-a10c-2ccf35534802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:26.985808 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbddcee-10b1-4a03-a10c-2ccf35534802 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:26.996749 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1325/2676] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:13:27.001532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb965b7-3b94-42d0-bddc-e5cd56ffe5dd admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:27.011717 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb965b7-3b94-42d0-bddc-e5cd56ffe5dd admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:27.017027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba2288-4f59-45a2-810a-e029e3ad75c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:27.017694 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba2288-4f59-45a2-810a-e029e3ad75c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:27.017976 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba2288-4f59-45a2-810a-e029e3ad75c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:27.018252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba2288-4f59-45a2-810a-e029e3ad75c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:27.018982 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba2288-4f59-45a2-810a-e029e3ad75c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:27.020575 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba2288-4f59-45a2-810a-e029e3ad75c5 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:27.035044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1352/2677] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:13:27.039352 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbddcee-10b1-4a03-a10c-2ccf35534802 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:27.050705 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbddcee-10b1-4a03-a10c-2ccf35534802 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:27.055660 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c5750-29ea-453a-8c59-888b4aed746c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:27.056563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c5750-29ea-453a-8c59-888b4aed746c admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:27.056880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c5750-29ea-453a-8c59-888b4aed746c admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:27.057157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c5750-29ea-453a-8c59-888b4aed746c admin admin] PATH_INFO: `/v3/projects/b46992df6b784a7abd397b8f3aeb827a/users/624bdc53a8b34b46b5004532716b3985/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:27.074868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c5750-29ea-453a-8c59-888b4aed746c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b46992df6b784a7abd397b8f3aeb827a, user_id=624bdc53a8b34b46b5004532716b3985, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:27.077760 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c5750-29ea-453a-8c59-888b4aed746c admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:27.093798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1326/2678] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:13:27 2022] PUT /identity/v3/projects/b46992df6b784a7abd397b8f3aeb827a/users/624bdc53a8b34b46b5004532716b3985/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:13:27.099247 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f437d-f31a-4ee5-9c72-183c7e9b500c None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:27.099540 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f437d-f31a-4ee5-9c72-183c7e9b500c None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:27.099998 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f437d-f31a-4ee5-9c72-183c7e9b500c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:27.124953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-948f437d-f31a-4ee5-9c72-183c7e9b500c None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:13:27.125804 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-948f437d-f31a-4ee5-9c72-183c7e9b500c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:27.187712 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1353/2679] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:13:27.197547 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c5750-29ea-453a-8c59-888b4aed746c admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:27.199268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d95c5750-29ea-453a-8c59-888b4aed746c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:27.257118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c5750-29ea-453a-8c59-888b4aed746c admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:27.260678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02446b03-8bbe-40d3-950c-a290ce3af99e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:27.261346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02446b03-8bbe-40d3-950c-a290ce3af99e service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:27.261632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02446b03-8bbe-40d3-950c-a290ce3af99e service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:27.261914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02446b03-8bbe-40d3-950c-a290ce3af99e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:27.272018 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02446b03-8bbe-40d3-950c-a290ce3af99e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:27.273515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02446b03-8bbe-40d3-950c-a290ce3af99e service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:27.283728 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1327/2680] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:13:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:13:31.604936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0c48e-7b77-4ee4-8ab7-736c8e557e23 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:31.605776 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0c48e-7b77-4ee4-8ab7-736c8e557e23 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:31.606072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0c48e-7b77-4ee4-8ab7-736c8e557e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:31.635204 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-f0f0c48e-7b77-4ee4-8ab7-736c8e557e23 None None] MFA Rules not processed for user `624bdc53a8b34b46b5004532716b3985`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:13:31.636437 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-f0f0c48e-7b77-4ee4-8ab7-736c8e557e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:31.656329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:13:31.718466 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1354/2681] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:13:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:13:31.723064 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02446b03-8bbe-40d3-950c-a290ce3af99e service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:31.734253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02446b03-8bbe-40d3-950c-a290ce3af99e service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:31.738993 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f508f88-7a47-43d9-b247-165bc488299b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:31.739643 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f508f88-7a47-43d9-b247-165bc488299b admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:31.739928 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f508f88-7a47-43d9-b247-165bc488299b admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:31.740205 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f508f88-7a47-43d9-b247-165bc488299b admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:31.741037 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f508f88-7a47-43d9-b247-165bc488299b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:31.743012 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f508f88-7a47-43d9-b247-165bc488299b admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:31.770485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1328/2682] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:13:31 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:13:31.774497 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0c48e-7b77-4ee4-8ab7-736c8e557e23 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:31.788241 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0c48e-7b77-4ee4-8ab7-736c8e557e23 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:31.860584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c65e07-2b40-40c2-bf3e-84f8754e240d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:31.861641 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c65e07-2b40-40c2-bf3e-84f8754e240d admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:31.861879 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c65e07-2b40-40c2-bf3e-84f8754e240d admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:31.862687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c65e07-2b40-40c2-bf3e-84f8754e240d admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:31.863215 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c65e07-2b40-40c2-bf3e-84f8754e240d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:31.865434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c65e07-2b40-40c2-bf3e-84f8754e240d admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:31.890485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1355/2683] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:13:31 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:13:31.894584 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f508f88-7a47-43d9-b247-165bc488299b admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:31.906602 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f508f88-7a47-43d9-b247-165bc488299b admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:31.911550 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fdea82-845f-4bec-8161-44f0035097cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:31.912405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdea82-845f-4bec-8161-44f0035097cb admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:31.912696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdea82-845f-4bec-8161-44f0035097cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:31.912696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdea82-845f-4bec-8161-44f0035097cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:31.912696 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fdea82-845f-4bec-8161-44f0035097cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:31.915529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fdea82-845f-4bec-8161-44f0035097cb admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:31.931338 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1329/2684] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:13:31.936004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c65e07-2b40-40c2-bf3e-84f8754e240d admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:31.949380 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c65e07-2b40-40c2-bf3e-84f8754e240d admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:31.954520 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00406ca-377f-499a-bd7f-0b61fa4ac7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:31.955425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00406ca-377f-499a-bd7f-0b61fa4ac7b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:31.955711 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00406ca-377f-499a-bd7f-0b61fa4ac7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:31.955993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00406ca-377f-499a-bd7f-0b61fa4ac7b2 admin admin] PATH_INFO: `/v3/projects/f5ecc2b62def4a579ad605653b167096/users/8a14af7da1bd49e7bc3dd258a42ad010/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:31.984199 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00406ca-377f-499a-bd7f-0b61fa4ac7b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ecc2b62def4a579ad605653b167096, user_id=8a14af7da1bd49e7bc3dd258a42ad010, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:31.986907 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00406ca-377f-499a-bd7f-0b61fa4ac7b2 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:32.003449 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1356/2685] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:13:31 2022] PUT /identity/v3/projects/f5ecc2b62def4a579ad605653b167096/users/8a14af7da1bd49e7bc3dd258a42ad010/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:13:32.007476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fdea82-845f-4bec-8161-44f0035097cb admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:32.019062 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fdea82-845f-4bec-8161-44f0035097cb admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:32.102893 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82be2f4-c03e-434d-a00a-dd7bbef25b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:32.103593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82be2f4-c03e-434d-a00a-dd7bbef25b5c admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:32.103882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82be2f4-c03e-434d-a00a-dd7bbef25b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:32.104165 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82be2f4-c03e-434d-a00a-dd7bbef25b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:32.105199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82be2f4-c03e-434d-a00a-dd7bbef25b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:32.107880 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82be2f4-c03e-434d-a00a-dd7bbef25b5c admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:32.119869 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1330/2686] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:13:32.124870 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00406ca-377f-499a-bd7f-0b61fa4ac7b2 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:32.137381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00406ca-377f-499a-bd7f-0b61fa4ac7b2 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:32.142865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579c60a-c6ed-4d3d-94bc-919f9f7edb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:32.143773 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579c60a-c6ed-4d3d-94bc-919f9f7edb5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:32.144058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579c60a-c6ed-4d3d-94bc-919f9f7edb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:32.144337 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579c60a-c6ed-4d3d-94bc-919f9f7edb5e admin admin] PATH_INFO: `/v3/projects/f5ecc2b62def4a579ad605653b167096/users/8a14af7da1bd49e7bc3dd258a42ad010/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:32.146417 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579c60a-c6ed-4d3d-94bc-919f9f7edb5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ecc2b62def4a579ad605653b167096, user_id=8a14af7da1bd49e7bc3dd258a42ad010, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:32.149349 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579c60a-c6ed-4d3d-94bc-919f9f7edb5e admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:32.160683 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1357/2687] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:13:32 2022] PUT /identity/v3/projects/f5ecc2b62def4a579ad605653b167096/users/8a14af7da1bd49e7bc3dd258a42ad010/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:13:37.031196 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3cded-0857-470d-9eaf-2f1f7038a1ed None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:37.031761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3cded-0857-470d-9eaf-2f1f7038a1ed None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:37.031859 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3cded-0857-470d-9eaf-2f1f7038a1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:37.079163 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-b4f3cded-0857-470d-9eaf-2f1f7038a1ed None None] MFA Rules not processed for user `8a14af7da1bd49e7bc3dd258a42ad010`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:13:37.080490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b4f3cded-0857-470d-9eaf-2f1f7038a1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:37.100889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:13:37.174892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1331/2688] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:13:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:13:37.184304 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579c60a-c6ed-4d3d-94bc-919f9f7edb5e admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:37.186171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-f579c60a-c6ed-4d3d-94bc-919f9f7edb5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:37.286043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579c60a-c6ed-4d3d-94bc-919f9f7edb5e admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:37.291977 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc6df1-98c7-4d41-8b82-6d8e29f4ffaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:37.292637 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc6df1-98c7-4d41-8b82-6d8e29f4ffaa service nova] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:37.292956 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc6df1-98c7-4d41-8b82-6d8e29f4ffaa service nova] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:37.293239 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc6df1-98c7-4d41-8b82-6d8e29f4ffaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:37.300275 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:13:37.354960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc6df1-98c7-4d41-8b82-6d8e29f4ffaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:37.357333 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc6df1-98c7-4d41-8b82-6d8e29f4ffaa service nova] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:37.368791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1358/2689] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:13:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:13:38.565639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694dfdb-9c7b-4441-88fb-2aa08b3016f6 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:38.566251 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694dfdb-9c7b-4441-88fb-2aa08b3016f6 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:38.566409 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694dfdb-9c7b-4441-88fb-2aa08b3016f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:38.590254 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-2694dfdb-9c7b-4441-88fb-2aa08b3016f6 None None] MFA Rules not processed for user `624bdc53a8b34b46b5004532716b3985`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:13:38.592336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-2694dfdb-9c7b-4441-88fb-2aa08b3016f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:38.600089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1332/2690] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:13:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:13:47.363633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=24 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:13:48.582966 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=23 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:13:53.549735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc6df1-98c7-4d41-8b82-6d8e29f4ffaa service nova] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:13:53.551231 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-7adc6df1-98c7-4d41-8b82-6d8e29f4ffaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:53.716898 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc6df1-98c7-4d41-8b82-6d8e29f4ffaa service nova] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:13:53.723013 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5715c-a1e6-4b37-af5e-3089e479c90a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f1ef52f8eb42a59a50a69873b9ffaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:13:53.724098 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5715c-a1e6-4b37-af5e-3089e479c90a service placement] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:13:53.724577 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5715c-a1e6-4b37-af5e-3089e479c90a service placement] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:13:53.725102 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5715c-a1e6-4b37-af5e-3089e479c90a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:13:53.727273 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-35a5715c-a1e6-4b37-af5e-3089e479c90a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:13:53.740415 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a5715c-a1e6-4b37-af5e-3089e479c90a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:13:53.741875 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a5715c-a1e6-4b37-af5e-3089e479c90a service placement] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:13:53.753264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1359/2691] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:13:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:14:03.748241 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=63 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:14:09.460557 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694dfdb-9c7b-4441-88fb-2aa08b3016f6 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:14:09.462020 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-2694dfdb-9c7b-4441-88fb-2aa08b3016f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:14:09.553384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694dfdb-9c7b-4441-88fb-2aa08b3016f6 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:14:09.557388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6988432-949b-476a-9628-232b647ad55e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:14:09.558042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6988432-949b-476a-9628-232b647ad55e service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:14:09.558330 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6988432-949b-476a-9628-232b647ad55e service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:14:09.558657 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6988432-949b-476a-9628-232b647ad55e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:14:09.569703 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6988432-949b-476a-9628-232b647ad55e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:14:09.571394 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6988432-949b-476a-9628-232b647ad55e service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:14:09.582583 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1333/2692] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:14:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:14:19.578010 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=41 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:15:19.755515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22458ac3-b2e1-452c-bf0e-61761c3fc593 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:15:19.756058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22458ac3-b2e1-452c-bf0e-61761c3fc593 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:15:19.756058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22458ac3-b2e1-452c-bf0e-61761c3fc593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:15:19.773575 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-22458ac3-b2e1-452c-bf0e-61761c3fc593 None None] MFA Rules not processed for user `24a41c09f2e4487f9897c9888a0d752f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:15:19.774789 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-22458ac3-b2e1-452c-bf0e-61761c3fc593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:15:19.833138 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1360/2693] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:15:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:15:20.155339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6988432-949b-476a-9628-232b647ad55e service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:15:20.164001 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6988432-949b-476a-9628-232b647ad55e service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:15:20.168422 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c6f4f-bd29-401e-954b-438a4ca81628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:15:20.169088 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c6f4f-bd29-401e-954b-438a4ca81628 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:15:20.169373 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c6f4f-bd29-401e-954b-438a4ca81628 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:15:20.169650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c6f4f-bd29-401e-954b-438a4ca81628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:15:20.171450 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-124c6f4f-bd29-401e-954b-438a4ca81628 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:15:20.181194 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c6f4f-bd29-401e-954b-438a4ca81628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:15:20.182659 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c6f4f-bd29-401e-954b-438a4ca81628 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:15:20.191209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1334/2694] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:15:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:15:29.826265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=25 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:15:30.187615 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=6 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:15:34.535986 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be2594-ba75-4ea3-8911-3f21cf2e858e None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:15:34.536788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be2594-ba75-4ea3-8911-3f21cf2e858e None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:15:34.536788 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be2594-ba75-4ea3-8911-3f21cf2e858e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:15:34.555154 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-67be2594-ba75-4ea3-8911-3f21cf2e858e None None] MFA Rules not processed for user `2d0d387beaf842c3b5978210e504fbed`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:15:34.556358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-67be2594-ba75-4ea3-8911-3f21cf2e858e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:15:34.611668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1361/2695] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:15:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:15:42.396697 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c6f4f-bd29-401e-954b-438a4ca81628 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:15:42.405525 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c6f4f-bd29-401e-954b-438a4ca81628 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:15:42.409268 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafdd5f-f0a9-41cc-a4e1-fe2b062e2288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:15:42.409906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafdd5f-f0a9-41cc-a4e1-fe2b062e2288 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:15:42.410191 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafdd5f-f0a9-41cc-a4e1-fe2b062e2288 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:15:42.410471 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafdd5f-f0a9-41cc-a4e1-fe2b062e2288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:15:42.412391 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-faafdd5f-f0a9-41cc-a4e1-fe2b062e2288 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:15:42.449937 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafdd5f-f0a9-41cc-a4e1-fe2b062e2288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:15:42.451424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafdd5f-f0a9-41cc-a4e1-fe2b062e2288 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:15:42.471652 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1335/2696] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:15:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:15:44.604452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=25 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:15:44.815986 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be2594-ba75-4ea3-8911-3f21cf2e858e None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:15:44.826375 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be2594-ba75-4ea3-8911-3f21cf2e858e None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:15:44.829970 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c53de-2e53-47b2-a63e-5ed30724aab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:15:44.830610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c53de-2e53-47b2-a63e-5ed30724aab8 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:15:44.830892 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c53de-2e53-47b2-a63e-5ed30724aab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:15:44.831171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c53de-2e53-47b2-a63e-5ed30724aab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:15:44.832983 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-090c53de-2e53-47b2-a63e-5ed30724aab8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:15:44.846846 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c53de-2e53-47b2-a63e-5ed30724aab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:15:44.848271 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c53de-2e53-47b2-a63e-5ed30724aab8 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:15:44.859299 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1362/2697] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:15:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:15:52.468884 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=18 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:15:54.854348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=6 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:16:18.551829 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafdd5f-f0a9-41cc-a4e1-fe2b062e2288 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:18.561337 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafdd5f-f0a9-41cc-a4e1-fe2b062e2288 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:18.565088 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e7d05-6dda-41bf-a6fb-14976b3ff250 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:18.565738 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e7d05-6dda-41bf-a6fb-14976b3ff250 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:18.566018 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e7d05-6dda-41bf-a6fb-14976b3ff250 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:18.566303 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e7d05-6dda-41bf-a6fb-14976b3ff250 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:18.568323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ee9e7d05-6dda-41bf-a6fb-14976b3ff250 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:18.578028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e7d05-6dda-41bf-a6fb-14976b3ff250 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:18.579452 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e7d05-6dda-41bf-a6fb-14976b3ff250 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:18.587459 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1336/2698] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:16:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:16:21.504666 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c53de-2e53-47b2-a63e-5ed30724aab8 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:21.515094 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c53de-2e53-47b2-a63e-5ed30724aab8 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:21.519052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37813e-6a88-4c0c-a25d-cce96a48c7dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:21.519834 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37813e-6a88-4c0c-a25d-cce96a48c7dd service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:21.520088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37813e-6a88-4c0c-a25d-cce96a48c7dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:21.520369 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37813e-6a88-4c0c-a25d-cce96a48c7dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:21.522225 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-bf37813e-6a88-4c0c-a25d-cce96a48c7dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:21.532648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37813e-6a88-4c0c-a25d-cce96a48c7dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:21.534253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37813e-6a88-4c0c-a25d-cce96a48c7dd service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:21.542693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1363/2699] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:16:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:16:21.615186 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e7d05-6dda-41bf-a6fb-14976b3ff250 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:21.623968 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e7d05-6dda-41bf-a6fb-14976b3ff250 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:21.628007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eec66b-1274-4527-b88c-4df80cd75078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:21.628651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eec66b-1274-4527-b88c-4df80cd75078 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:21.628971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eec66b-1274-4527-b88c-4df80cd75078 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:21.629252 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eec66b-1274-4527-b88c-4df80cd75078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:21.631116 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-06eec66b-1274-4527-b88c-4df80cd75078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:21.645628 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eec66b-1274-4527-b88c-4df80cd75078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:21.647058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eec66b-1274-4527-b88c-4df80cd75078 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:21.659258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1337/2700] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:16:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:16:21.721490 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37813e-6a88-4c0c-a25d-cce96a48c7dd service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:21.722965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-bf37813e-6a88-4c0c-a25d-cce96a48c7dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:21.738358 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37813e-6a88-4c0c-a25d-cce96a48c7dd service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:21.744009 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1cb266-bf80-40a1-8e41-3049e272055d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f1ef52f8eb42a59a50a69873b9ffaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:21.744647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cb266-bf80-40a1-8e41-3049e272055d service placement] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:21.744965 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cb266-bf80-40a1-8e41-3049e272055d service placement] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:21.745248 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cb266-bf80-40a1-8e41-3049e272055d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:21.746985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ab1cb266-bf80-40a1-8e41-3049e272055d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:21.758434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1cb266-bf80-40a1-8e41-3049e272055d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:21.760002 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1cb266-bf80-40a1-8e41-3049e272055d service placement] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:21.770420 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1364/2701] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:16:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:16:31.653132 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=12 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:16:31.766436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=12 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:16:47.758786 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eec66b-1274-4527-b88c-4df80cd75078 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:47.761204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-06eec66b-1274-4527-b88c-4df80cd75078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:47.778136 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eec66b-1274-4527-b88c-4df80cd75078 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:47.787881 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dd918-5f31-468f-86ec-4656fafc6c03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:47.788651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dd918-5f31-468f-86ec-4656fafc6c03 service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:47.789293 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dd918-5f31-468f-86ec-4656fafc6c03 service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:47.789645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dd918-5f31-468f-86ec-4656fafc6c03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:47.791681 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c01dd918-5f31-468f-86ec-4656fafc6c03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:47.809251 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01dd918-5f31-468f-86ec-4656fafc6c03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:47.810874 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01dd918-5f31-468f-86ec-4656fafc6c03 service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:47.823791 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1338/2702] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:16:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:16:55.654083 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1cb266-bf80-40a1-8e41-3049e272055d service placement] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:55.673117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1cb266-bf80-40a1-8e41-3049e272055d service placement] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:55.681478 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e56dac-9a48-47a4-8251-121ed1369839 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f1ef52f8eb42a59a50a69873b9ffaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:55.686159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e56dac-9a48-47a4-8251-121ed1369839 service placement] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:55.686795 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e56dac-9a48-47a4-8251-121ed1369839 service placement] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:55.688008 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e56dac-9a48-47a4-8251-121ed1369839 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:55.691772 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-60e56dac-9a48-47a4-8251-121ed1369839 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:55.713831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e56dac-9a48-47a4-8251-121ed1369839 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:55.716138 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e56dac-9a48-47a4-8251-121ed1369839 service placement] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:55.731051 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dd918-5f31-468f-86ec-4656fafc6c03 service nova] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:55.731592 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c01dd918-5f31-468f-86ec-4656fafc6c03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:55.754863 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1365/2703] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:16:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:16:55.861328 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dd918-5f31-468f-86ec-4656fafc6c03 service nova] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:55.865968 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7d4c7-f85b-4885-8c5d-e13915e20854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:55.866843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7d4c7-f85b-4885-8c5d-e13915e20854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:55.867129 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7d4c7-f85b-4885-8c5d-e13915e20854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:55.867405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7d4c7-f85b-4885-8c5d-e13915e20854 admin admin] PATH_INFO: `/v3/users/2d0d387beaf842c3b5978210e504fbed` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:55.868666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7d4c7-f85b-4885-8c5d-e13915e20854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d0d387beaf842c3b5978210e504fbed)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:55.870617 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7d4c7-f85b-4885-8c5d-e13915e20854 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:55.920357 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b6d7d4c7-f85b-4885-8c5d-e13915e20854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d0d387beaf842c3b5978210e504fbed'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:55.927842 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b6d7d4c7-f85b-4885-8c5d-e13915e20854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d0d387beaf842c3b5978210e504fbed'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:55.936857 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b6d7d4c7-f85b-4885-8c5d-e13915e20854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d0d387beaf842c3b5978210e504fbed'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:55.938380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b6d7d4c7-f85b-4885-8c5d-e13915e20854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d0d387beaf842c3b5978210e504fbed'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:55.939156 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-b6d7d4c7-f85b-4885-8c5d-e13915e20854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d0d387beaf842c3b5978210e504fbed'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:55.966241 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1339/2704] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:16:55 2022] DELETE /identity/v3/users/2d0d387beaf842c3b5978210e504fbed => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:16:56.181586 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e56dac-9a48-47a4-8251-121ed1369839 service placement] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:56.197646 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e56dac-9a48-47a4-8251-121ed1369839 service placement] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:56.271980 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c02bc-a88b-4129-a35e-942351b40d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:56.272892 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c02bc-a88b-4129-a35e-942351b40d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:56.273173 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c02bc-a88b-4129-a35e-942351b40d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:56.273451 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c02bc-a88b-4129-a35e-942351b40d5e admin admin] PATH_INFO: `/v3/projects/b80ca69f6cd7402cb644c7721015ebc8` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:56.274639 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c02bc-a88b-4129-a35e-942351b40d5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b80ca69f6cd7402cb644c7721015ebc8)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:56.277301 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c02bc-a88b-4129-a35e-942351b40d5e admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:56.337047 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ce3c02bc-a88b-4129-a35e-942351b40d5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b80ca69f6cd7402cb644c7721015ebc8', 'request_id': 'req-ce3c02bc-a88b-4129-a35e-942351b40d5e'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:56.342640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ce3c02bc-a88b-4129-a35e-942351b40d5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b80ca69f6cd7402cb644c7721015ebc8', 'request_id': 'req-ce3c02bc-a88b-4129-a35e-942351b40d5e'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:56.352803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ce3c02bc-a88b-4129-a35e-942351b40d5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b80ca69f6cd7402cb644c7721015ebc8', 'request_id': 'req-ce3c02bc-a88b-4129-a35e-942351b40d5e'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:56.365807 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ce3c02bc-a88b-4129-a35e-942351b40d5e admin admin] The token cache is being invalidate because project b80ca69f6cd7402cb644c7721015ebc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:16:56.366143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ce3c02bc-a88b-4129-a35e-942351b40d5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:56.366861 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ce3c02bc-a88b-4129-a35e-942351b40d5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:56.368551 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1366/2705] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:16:56 2022] DELETE /identity/v3/projects/b80ca69f6cd7402cb644c7721015ebc8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:16:56.375760 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36072f9b-74aa-4659-a0d1-3fa2b4dc7482 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:56.376087 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36072f9b-74aa-4659-a0d1-3fa2b4dc7482 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:56.378076 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36072f9b-74aa-4659-a0d1-3fa2b4dc7482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:56.469916 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:16:56.473888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-36072f9b-74aa-4659-a0d1-3fa2b4dc7482 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:16:56.476254 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-36072f9b-74aa-4659-a0d1-3fa2b4dc7482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:56.545240 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1340/2706] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:16:56.553680 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231a76c-63ec-43ad-bbc1-453b49e6af25 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:56.554018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231a76c-63ec-43ad-bbc1-453b49e6af25 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:56.554235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231a76c-63ec-43ad-bbc1-453b49e6af25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:56.581645 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-9231a76c-63ec-43ad-bbc1-453b49e6af25 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:16:56.584598 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-9231a76c-63ec-43ad-bbc1-453b49e6af25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:56.593198 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1367/2707] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:16:56.598170 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36072f9b-74aa-4659-a0d1-3fa2b4dc7482 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:56.609992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36072f9b-74aa-4659-a0d1-3fa2b4dc7482 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:56.617186 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be493b08-1be7-4461-b0cf-616d3d38ceb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:56.617881 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be493b08-1be7-4461-b0cf-616d3d38ceb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:56.618163 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be493b08-1be7-4461-b0cf-616d3d38ceb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:56.618438 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be493b08-1be7-4461-b0cf-616d3d38ceb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:56.619169 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be493b08-1be7-4461-b0cf-616d3d38ceb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:16:56.619476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be493b08-1be7-4461-b0cf-616d3d38ceb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:56.621193 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be493b08-1be7-4461-b0cf-616d3d38ceb0 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:56.637789 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1341/2708] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:16:56.643398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231a76c-63ec-43ad-bbc1-453b49e6af25 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:56.653700 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231a76c-63ec-43ad-bbc1-453b49e6af25 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:56.659911 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dbc67-01d8-499b-8a78-8dd4a1b52702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:56.660633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dbc67-01d8-499b-8a78-8dd4a1b52702 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:56.660778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dbc67-01d8-499b-8a78-8dd4a1b52702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:56.661256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dbc67-01d8-499b-8a78-8dd4a1b52702 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:56.662556 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236dbc67-01d8-499b-8a78-8dd4a1b52702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:56.665391 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236dbc67-01d8-499b-8a78-8dd4a1b52702 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:56.710930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1368/2709] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:16:56 2022] POST /identity/v3/projects => generated 405 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:16:56.715022 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be493b08-1be7-4461-b0cf-616d3d38ceb0 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:56.726299 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be493b08-1be7-4461-b0cf-616d3d38ceb0 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:56.788539 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed692585-cd90-402b-a6f8-93ed44520d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:56.789270 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed692585-cd90-402b-a6f8-93ed44520d0d admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:56.789527 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed692585-cd90-402b-a6f8-93ed44520d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:56.789807 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed692585-cd90-402b-a6f8-93ed44520d0d admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:56.790547 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed692585-cd90-402b-a6f8-93ed44520d0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:56.792485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed692585-cd90-402b-a6f8-93ed44520d0d admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:56.812517 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1342/2710] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:16:56 2022] POST /identity/v3/users => generated 303 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:16:56.817717 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dbc67-01d8-499b-8a78-8dd4a1b52702 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:56.828270 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dbc67-01d8-499b-8a78-8dd4a1b52702 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:56.835003 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04491dcc-d113-4bde-8bcf-acc49dbb3a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:56.835756 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04491dcc-d113-4bde-8bcf-acc49dbb3a95 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:56.836083 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04491dcc-d113-4bde-8bcf-acc49dbb3a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:56.836365 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04491dcc-d113-4bde-8bcf-acc49dbb3a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:56.837213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04491dcc-d113-4bde-8bcf-acc49dbb3a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:56.838837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04491dcc-d113-4bde-8bcf-acc49dbb3a95 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:56.849656 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1369/2711] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:16:56.853778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed692585-cd90-402b-a6f8-93ed44520d0d admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:56.864197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed692585-cd90-402b-a6f8-93ed44520d0d admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:56.869697 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8b143-eeda-4389-95ab-384f26e04c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:56.870573 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8b143-eeda-4389-95ab-384f26e04c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:56.870862 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8b143-eeda-4389-95ab-384f26e04c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:56.871147 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8b143-eeda-4389-95ab-384f26e04c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:56.871892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8b143-eeda-4389-95ab-384f26e04c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:56.873596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8b143-eeda-4389-95ab-384f26e04c61 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:56.885090 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1343/2712] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:16:56.889360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04491dcc-d113-4bde-8bcf-acc49dbb3a95 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:56.900470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04491dcc-d113-4bde-8bcf-acc49dbb3a95 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:56.906284 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851edb52-bc3e-4b11-b57b-72b33b17ef53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:56.907387 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851edb52-bc3e-4b11-b57b-72b33b17ef53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:56.907755 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851edb52-bc3e-4b11-b57b-72b33b17ef53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:56.908041 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851edb52-bc3e-4b11-b57b-72b33b17ef53 admin admin] PATH_INFO: `/v3/projects/9477b06774014b39893c366013336a5e/users/c4c669bab30445c8afa7760a9a06ad78/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:56.926365 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851edb52-bc3e-4b11-b57b-72b33b17ef53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9477b06774014b39893c366013336a5e, user_id=c4c669bab30445c8afa7760a9a06ad78, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:56.929124 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851edb52-bc3e-4b11-b57b-72b33b17ef53 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:56.944236 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1370/2713] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:16:56 2022] PUT /identity/v3/projects/9477b06774014b39893c366013336a5e/users/c4c669bab30445c8afa7760a9a06ad78/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:16:56.956471 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e754a18-2baa-4c40-a87c-3561c7a82752 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:56.965454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e754a18-2baa-4c40-a87c-3561c7a82752 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:56.965861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e754a18-2baa-4c40-a87c-3561c7a82752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:56.993246 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-8e754a18-2baa-4c40-a87c-3561c7a82752 None None] MFA Rules not processed for user `c4c669bab30445c8afa7760a9a06ad78`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:16:56.994486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8e754a18-2baa-4c40-a87c-3561c7a82752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:57.067859 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1344/2714] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:16:56 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:16:57.068530 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851edb52-bc3e-4b11-b57b-72b33b17ef53 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:57.075055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:16:57.144054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851edb52-bc3e-4b11-b57b-72b33b17ef53 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:57.149209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18276d64-ebfd-43ce-9647-f97e0669a71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:57.150108 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18276d64-ebfd-43ce-9647-f97e0669a71b admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:57.150390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18276d64-ebfd-43ce-9647-f97e0669a71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:57.150663 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18276d64-ebfd-43ce-9647-f97e0669a71b admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:57.151534 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18276d64-ebfd-43ce-9647-f97e0669a71b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:57.153431 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18276d64-ebfd-43ce-9647-f97e0669a71b admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:57.184013 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1371/2715] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:16:57 2022] POST /identity/v3/projects => generated 405 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:16:57.185506 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e754a18-2baa-4c40-a87c-3561c7a82752 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:57.191721 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:16:57.198899 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e754a18-2baa-4c40-a87c-3561c7a82752 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:57.268434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855434a-143f-4c10-8241-77083a8144bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:57.269150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855434a-143f-4c10-8241-77083a8144bf admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:57.269438 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855434a-143f-4c10-8241-77083a8144bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:57.269719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855434a-143f-4c10-8241-77083a8144bf admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:57.270408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855434a-143f-4c10-8241-77083a8144bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:57.272329 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855434a-143f-4c10-8241-77083a8144bf admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:57.295610 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1345/2716] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:16:57 2022] POST /identity/v3/users => generated 309 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:16:57.298044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18276d64-ebfd-43ce-9647-f97e0669a71b admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:57.308098 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18276d64-ebfd-43ce-9647-f97e0669a71b admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:57.312929 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec569f-1f3d-4a70-98e9-db20f990ac5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:57.314235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec569f-1f3d-4a70-98e9-db20f990ac5c admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:57.314521 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec569f-1f3d-4a70-98e9-db20f990ac5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:57.314803 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec569f-1f3d-4a70-98e9-db20f990ac5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:57.315546 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec569f-1f3d-4a70-98e9-db20f990ac5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:57.317549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec569f-1f3d-4a70-98e9-db20f990ac5c admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:57.329452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1372/2717] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:16:57.333864 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855434a-143f-4c10-8241-77083a8144bf admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:57.344305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855434a-143f-4c10-8241-77083a8144bf admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:57.349476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573bd7b8-790c-4a58-a549-77f1e822db45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:57.351458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bd7b8-790c-4a58-a549-77f1e822db45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:57.352028 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bd7b8-790c-4a58-a549-77f1e822db45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:57.352678 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bd7b8-790c-4a58-a549-77f1e822db45 admin admin] PATH_INFO: `/v3/projects/2933dbb105f74df6b82d7344302ff6ff/users/9acb138f024941c8bd3b1b6a1ce4a85e/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:57.369676 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573bd7b8-790c-4a58-a549-77f1e822db45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2933dbb105f74df6b82d7344302ff6ff, user_id=9acb138f024941c8bd3b1b6a1ce4a85e, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:57.372358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573bd7b8-790c-4a58-a549-77f1e822db45 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:57.388759 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1346/2718] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:16:57 2022] PUT /identity/v3/projects/2933dbb105f74df6b82d7344302ff6ff/users/9acb138f024941c8bd3b1b6a1ce4a85e/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:16:57.393112 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec569f-1f3d-4a70-98e9-db20f990ac5c admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:57.403913 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec569f-1f3d-4a70-98e9-db20f990ac5c admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:57.469856 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb4fcc-d7f3-4ce3-92db-01a68a8b51fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:57.470657 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb4fcc-d7f3-4ce3-92db-01a68a8b51fe admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:57.470949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb4fcc-d7f3-4ce3-92db-01a68a8b51fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:57.471610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb4fcc-d7f3-4ce3-92db-01a68a8b51fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:57.472027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb4fcc-d7f3-4ce3-92db-01a68a8b51fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:57.473767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb4fcc-d7f3-4ce3-92db-01a68a8b51fe admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:57.485479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1373/2719] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:16:57.490070 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573bd7b8-790c-4a58-a549-77f1e822db45 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:57.503200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573bd7b8-790c-4a58-a549-77f1e822db45 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:57.508016 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043e3df-6fd3-40f3-832c-c1fe3fc968d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:57.509031 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043e3df-6fd3-40f3-832c-c1fe3fc968d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:57.509317 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043e3df-6fd3-40f3-832c-c1fe3fc968d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:57.509598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043e3df-6fd3-40f3-832c-c1fe3fc968d6 admin admin] PATH_INFO: `/v3/projects/2933dbb105f74df6b82d7344302ff6ff/users/9acb138f024941c8bd3b1b6a1ce4a85e/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:57.511681 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043e3df-6fd3-40f3-832c-c1fe3fc968d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2933dbb105f74df6b82d7344302ff6ff, user_id=9acb138f024941c8bd3b1b6a1ce4a85e, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:57.514580 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043e3df-6fd3-40f3-832c-c1fe3fc968d6 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:57.525664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1347/2720] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:16:57 2022] PUT /identity/v3/projects/2933dbb105f74df6b82d7344302ff6ff/users/9acb138f024941c8bd3b1b6a1ce4a85e/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:16:57.536433 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011415a3-e114-403a-82f1-8388eae7e019 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:57.536758 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011415a3-e114-403a-82f1-8388eae7e019 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:57.537104 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011415a3-e114-403a-82f1-8388eae7e019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:57.566837 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-011415a3-e114-403a-82f1-8388eae7e019 None None] MFA Rules not processed for user `9acb138f024941c8bd3b1b6a1ce4a85e`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:16:57.568327 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-011415a3-e114-403a-82f1-8388eae7e019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:57.636142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:16:57.641377 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1374/2721] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:16:57 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:16:57.651634 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043e3df-6fd3-40f3-832c-c1fe3fc968d6 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:57.653392 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a043e3df-6fd3-40f3-832c-c1fe3fc968d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:57.737719 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043e3df-6fd3-40f3-832c-c1fe3fc968d6 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:57.743393 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a8295-61de-42bd-b02a-7bb3c7be4949 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:57.744286 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a8295-61de-42bd-b02a-7bb3c7be4949 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:57.744599 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a8295-61de-42bd-b02a-7bb3c7be4949 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:57.745014 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a8295-61de-42bd-b02a-7bb3c7be4949 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:57.758055 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a8295-61de-42bd-b02a-7bb3c7be4949 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:57.759529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a8295-61de-42bd-b02a-7bb3c7be4949 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:57.770803 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1348/2722] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:16:57 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:16:57.914119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011415a3-e114-403a-82f1-8388eae7e019 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:57.979943 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011415a3-e114-403a-82f1-8388eae7e019 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:57.984496 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb90e67-e9e2-4e04-acdd-a3058b238b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:57.985416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb90e67-e9e2-4e04-acdd-a3058b238b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:57.986340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb90e67-e9e2-4e04-acdd-a3058b238b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:57.986629 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb90e67-e9e2-4e04-acdd-a3058b238b5b admin admin] PATH_INFO: `/v3/users/c4c669bab30445c8afa7760a9a06ad78` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:57.987820 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb90e67-e9e2-4e04-acdd-a3058b238b5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4c669bab30445c8afa7760a9a06ad78)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:57.989785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb90e67-e9e2-4e04-acdd-a3058b238b5b admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:58.036974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-cdb90e67-e9e2-4e04-acdd-a3058b238b5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4c669bab30445c8afa7760a9a06ad78'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:58.049502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-cdb90e67-e9e2-4e04-acdd-a3058b238b5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4c669bab30445c8afa7760a9a06ad78'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:58.056691 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-cdb90e67-e9e2-4e04-acdd-a3058b238b5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4c669bab30445c8afa7760a9a06ad78'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:58.064883 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-cdb90e67-e9e2-4e04-acdd-a3058b238b5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4c669bab30445c8afa7760a9a06ad78'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:58.065507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-cdb90e67-e9e2-4e04-acdd-a3058b238b5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4c669bab30445c8afa7760a9a06ad78'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:58.071665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1375/2723] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:16:57 2022] DELETE /identity/v3/users/c4c669bab30445c8afa7760a9a06ad78 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:16:58.076318 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ea6b0-020f-4c6a-a1ec-77c4cf5d37d6 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:58.076874 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ea6b0-020f-4c6a-a1ec-77c4cf5d37d6 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:58.076981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ea6b0-020f-4c6a-a1ec-77c4cf5d37d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:58.094142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-984ea6b0-020f-4c6a-a1ec-77c4cf5d37d6 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:16:58.095540 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-984ea6b0-020f-4c6a-a1ec-77c4cf5d37d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:58.129792 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:16:58.161168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1349/2724] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:16:58.170764 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb90e67-e9e2-4e04-acdd-a3058b238b5b admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:58.172226 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-cdb90e67-e9e2-4e04-acdd-a3058b238b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:58.230300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb90e67-e9e2-4e04-acdd-a3058b238b5b admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:58.234053 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f04d36-d449-463c-9c7c-de80755f9615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:58.234697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f04d36-d449-463c-9c7c-de80755f9615 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:58.234989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f04d36-d449-463c-9c7c-de80755f9615 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:58.235305 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f04d36-d449-463c-9c7c-de80755f9615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:58.245532 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f04d36-d449-463c-9c7c-de80755f9615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:58.246968 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f04d36-d449-463c-9c7c-de80755f9615 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:58.256578 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1376/2725] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:16:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:16:58.519304 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ea6b0-020f-4c6a-a1ec-77c4cf5d37d6 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:58.520854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-984ea6b0-020f-4c6a-a1ec-77c4cf5d37d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:58.534744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ea6b0-020f-4c6a-a1ec-77c4cf5d37d6 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:58.539708 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af62d9-92b0-4d85-b50c-3b2e45379187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:58.541112 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af62d9-92b0-4d85-b50c-3b2e45379187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:58.541405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af62d9-92b0-4d85-b50c-3b2e45379187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:58.541860 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af62d9-92b0-4d85-b50c-3b2e45379187 admin admin] PATH_INFO: `/v3/projects/9477b06774014b39893c366013336a5e` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:58.543191 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af62d9-92b0-4d85-b50c-3b2e45379187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9477b06774014b39893c366013336a5e)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:58.545095 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af62d9-92b0-4d85-b50c-3b2e45379187 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:58.594294 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-e6af62d9-92b0-4d85-b50c-3b2e45379187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9477b06774014b39893c366013336a5e', 'request_id': 'req-e6af62d9-92b0-4d85-b50c-3b2e45379187'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:58.601159 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-e6af62d9-92b0-4d85-b50c-3b2e45379187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9477b06774014b39893c366013336a5e', 'request_id': 'req-e6af62d9-92b0-4d85-b50c-3b2e45379187'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:58.608842 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-e6af62d9-92b0-4d85-b50c-3b2e45379187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9477b06774014b39893c366013336a5e', 'request_id': 'req-e6af62d9-92b0-4d85-b50c-3b2e45379187'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:58.620620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-e6af62d9-92b0-4d85-b50c-3b2e45379187 admin admin] The token cache is being invalidate because project 9477b06774014b39893c366013336a5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:16:58.620945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-e6af62d9-92b0-4d85-b50c-3b2e45379187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:58.621586 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-e6af62d9-92b0-4d85-b50c-3b2e45379187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:58.623134 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1350/2726] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:16:58 2022] DELETE /identity/v3/projects/9477b06774014b39893c366013336a5e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:16:58.627324 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f04d36-d449-463c-9c7c-de80755f9615 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:58.637055 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f04d36-d449-463c-9c7c-de80755f9615 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:58.655864 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:16:58.697197 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed44e29-2ffa-49c0-a4fb-211fb06874ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:58.698064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed44e29-2ffa-49c0-a4fb-211fb06874ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:58.698342 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed44e29-2ffa-49c0-a4fb-211fb06874ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:58.698615 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed44e29-2ffa-49c0-a4fb-211fb06874ca admin admin] PATH_INFO: `/v3/users/9acb138f024941c8bd3b1b6a1ce4a85e` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:58.699777 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed44e29-2ffa-49c0-a4fb-211fb06874ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=9acb138f024941c8bd3b1b6a1ce4a85e)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:58.701752 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed44e29-2ffa-49c0-a4fb-211fb06874ca admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:58.748155 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2ed44e29-2ffa-49c0-a4fb-211fb06874ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9acb138f024941c8bd3b1b6a1ce4a85e'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:58.759995 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2ed44e29-2ffa-49c0-a4fb-211fb06874ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9acb138f024941c8bd3b1b6a1ce4a85e'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:58.767256 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2ed44e29-2ffa-49c0-a4fb-211fb06874ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9acb138f024941c8bd3b1b6a1ce4a85e'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:58.774540 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2ed44e29-2ffa-49c0-a4fb-211fb06874ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9acb138f024941c8bd3b1b6a1ce4a85e'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:58.775156 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2ed44e29-2ffa-49c0-a4fb-211fb06874ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9acb138f024941c8bd3b1b6a1ce4a85e'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:58.781087 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1377/2727] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:16:58 2022] DELETE /identity/v3/users/9acb138f024941c8bd3b1b6a1ce4a85e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:16:59.062258 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af62d9-92b0-4d85-b50c-3b2e45379187 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:59.064546 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-e6af62d9-92b0-4d85-b50c-3b2e45379187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:59.081359 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af62d9-92b0-4d85-b50c-3b2e45379187 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:59.137267 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd16585-b798-4fa7-8d30-01e983aa1095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:59.138138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd16585-b798-4fa7-8d30-01e983aa1095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:59.138424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd16585-b798-4fa7-8d30-01e983aa1095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:59.138705 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd16585-b798-4fa7-8d30-01e983aa1095 admin admin] PATH_INFO: `/v3/projects/2933dbb105f74df6b82d7344302ff6ff` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:59.139977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd16585-b798-4fa7-8d30-01e983aa1095 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2933dbb105f74df6b82d7344302ff6ff)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:59.142449 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd16585-b798-4fa7-8d30-01e983aa1095 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:59.189070 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-cbd16585-b798-4fa7-8d30-01e983aa1095 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2933dbb105f74df6b82d7344302ff6ff', 'request_id': 'req-cbd16585-b798-4fa7-8d30-01e983aa1095'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:59.195584 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-cbd16585-b798-4fa7-8d30-01e983aa1095 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2933dbb105f74df6b82d7344302ff6ff', 'request_id': 'req-cbd16585-b798-4fa7-8d30-01e983aa1095'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:59.202691 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-cbd16585-b798-4fa7-8d30-01e983aa1095 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2933dbb105f74df6b82d7344302ff6ff', 'request_id': 'req-cbd16585-b798-4fa7-8d30-01e983aa1095'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:59.205282 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:16:59.216372 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-cbd16585-b798-4fa7-8d30-01e983aa1095 admin admin] The token cache is being invalidate because project 2933dbb105f74df6b82d7344302ff6ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:16:59.216665 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-cbd16585-b798-4fa7-8d30-01e983aa1095 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:59.217323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-cbd16585-b798-4fa7-8d30-01e983aa1095 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:16:59.218865 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1351/2728] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:16:59 2022] DELETE /identity/v3/projects/2933dbb105f74df6b82d7344302ff6ff => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:16:59.223993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2029f9b-c902-46b7-8d16-48ec7e559a81 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:59.224467 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2029f9b-c902-46b7-8d16-48ec7e559a81 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:59.224583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2029f9b-c902-46b7-8d16-48ec7e559a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:59.242763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-b2029f9b-c902-46b7-8d16-48ec7e559a81 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:16:59.243987 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b2029f9b-c902-46b7-8d16-48ec7e559a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:59.304459 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1378/2729] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:16:59.312152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23803f3f-9fa8-446b-9327-53a6659cc591 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:59.312383 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23803f3f-9fa8-446b-9327-53a6659cc591 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:59.312668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23803f3f-9fa8-446b-9327-53a6659cc591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:59.332393 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-23803f3f-9fa8-446b-9327-53a6659cc591 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:16:59.333657 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-23803f3f-9fa8-446b-9327-53a6659cc591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:59.341940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1352/2730] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:16:59.346257 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2029f9b-c902-46b7-8d16-48ec7e559a81 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:59.355974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2029f9b-c902-46b7-8d16-48ec7e559a81 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:59.361113 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98e50b-42a1-4070-80eb-36796094b1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:59.361855 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98e50b-42a1-4070-80eb-36796094b1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:59.362141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98e50b-42a1-4070-80eb-36796094b1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:59.362419 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98e50b-42a1-4070-80eb-36796094b1d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:59.363145 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98e50b-42a1-4070-80eb-36796094b1d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:16:59.363453 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98e50b-42a1-4070-80eb-36796094b1d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:59.365105 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98e50b-42a1-4070-80eb-36796094b1d7 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:59.377315 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1379/2731] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:16:59.381578 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23803f3f-9fa8-446b-9327-53a6659cc591 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:59.390941 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23803f3f-9fa8-446b-9327-53a6659cc591 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:59.395694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cf9c8-ab5d-450e-a750-d40d0d028c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:59.396351 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cf9c8-ab5d-450e-a750-d40d0d028c58 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:59.396641 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cf9c8-ab5d-450e-a750-d40d0d028c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:59.396968 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cf9c8-ab5d-450e-a750-d40d0d028c58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:59.397684 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2cf9c8-ab5d-450e-a750-d40d0d028c58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:59.399604 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2cf9c8-ab5d-450e-a750-d40d0d028c58 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:59.426352 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1353/2732] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:16:59 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:16:59.430729 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98e50b-42a1-4070-80eb-36796094b1d7 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:59.441840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98e50b-42a1-4070-80eb-36796094b1d7 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:59.468333 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:16:59.501672 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d27a47-e7d9-40ab-a9f3-84fe43b034d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:59.502346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d27a47-e7d9-40ab-a9f3-84fe43b034d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:59.502633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d27a47-e7d9-40ab-a9f3-84fe43b034d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:59.502910 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d27a47-e7d9-40ab-a9f3-84fe43b034d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:59.503599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d27a47-e7d9-40ab-a9f3-84fe43b034d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:59.506231 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d27a47-e7d9-40ab-a9f3-84fe43b034d1 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:59.527896 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1380/2733] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:16:59 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:16:59.531975 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cf9c8-ab5d-450e-a750-d40d0d028c58 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:59.543259 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cf9c8-ab5d-450e-a750-d40d0d028c58 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:59.548495 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804e7c5-d873-43a9-b201-fdee6efacf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:59.549243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804e7c5-d873-43a9-b201-fdee6efacf14 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:59.549575 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804e7c5-d873-43a9-b201-fdee6efacf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:59.549858 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804e7c5-d873-43a9-b201-fdee6efacf14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:59.550588 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804e7c5-d873-43a9-b201-fdee6efacf14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:59.552198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804e7c5-d873-43a9-b201-fdee6efacf14 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:59.565240 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1354/2734] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:16:59.569495 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d27a47-e7d9-40ab-a9f3-84fe43b034d1 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:59.580290 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d27a47-e7d9-40ab-a9f3-84fe43b034d1 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:59.584991 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b715fd8-1a40-4513-a7a7-eee606a535b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:59.585654 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b715fd8-1a40-4513-a7a7-eee606a535b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:59.585945 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b715fd8-1a40-4513-a7a7-eee606a535b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:59.586243 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b715fd8-1a40-4513-a7a7-eee606a535b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:59.586961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b715fd8-1a40-4513-a7a7-eee606a535b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:59.588566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b715fd8-1a40-4513-a7a7-eee606a535b9 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:59.598949 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1381/2735] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:16:59.603087 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804e7c5-d873-43a9-b201-fdee6efacf14 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:59.613995 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804e7c5-d873-43a9-b201-fdee6efacf14 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:59.618971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01692b6-e61e-4cbc-91a4-c5059c662203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:59.619850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01692b6-e61e-4cbc-91a4-c5059c662203 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:59.620136 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01692b6-e61e-4cbc-91a4-c5059c662203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:59.620417 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01692b6-e61e-4cbc-91a4-c5059c662203 admin admin] PATH_INFO: `/v3/projects/d790e0a18cd84cc9815b9ef572835004/users/93a9db7f69854e7e8fd6366d5c8534c3/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:59.636639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01692b6-e61e-4cbc-91a4-c5059c662203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d790e0a18cd84cc9815b9ef572835004, user_id=93a9db7f69854e7e8fd6366d5c8534c3, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:59.639328 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01692b6-e61e-4cbc-91a4-c5059c662203 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:59.652359 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1355/2736] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:16:59 2022] PUT /identity/v3/projects/d790e0a18cd84cc9815b9ef572835004/users/93a9db7f69854e7e8fd6366d5c8534c3/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:16:59.664401 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8de507-8ec2-4ebb-ace7-2f4463438d0a None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:59.664689 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8de507-8ec2-4ebb-ace7-2f4463438d0a None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:59.664998 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8de507-8ec2-4ebb-ace7-2f4463438d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:59.693101 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-8b8de507-8ec2-4ebb-ace7-2f4463438d0a None None] MFA Rules not processed for user `93a9db7f69854e7e8fd6366d5c8534c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:16:59.694322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8b8de507-8ec2-4ebb-ace7-2f4463438d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:16:59.761390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1382/2737] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:16:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:16:59.767543 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01692b6-e61e-4cbc-91a4-c5059c662203 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:59.836849 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01692b6-e61e-4cbc-91a4-c5059c662203 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:59.841930 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149729a-9a3a-4b58-b820-bb9bd2d259cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:59.842594 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149729a-9a3a-4b58-b820-bb9bd2d259cc admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:59.842905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149729a-9a3a-4b58-b820-bb9bd2d259cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:59.843211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149729a-9a3a-4b58-b820-bb9bd2d259cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:59.843920 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149729a-9a3a-4b58-b820-bb9bd2d259cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:59.846570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149729a-9a3a-4b58-b820-bb9bd2d259cc admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:59.873512 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1356/2738] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:16:59 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:16:59.877828 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8de507-8ec2-4ebb-ace7-2f4463438d0a None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:59.888591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8de507-8ec2-4ebb-ace7-2f4463438d0a None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:59.945870 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb724b1-8776-4466-a6d8-2f0a2d06f9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:59.946537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb724b1-8776-4466-a6d8-2f0a2d06f9eb admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:59.946818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb724b1-8776-4466-a6d8-2f0a2d06f9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:59.947090 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb724b1-8776-4466-a6d8-2f0a2d06f9eb admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:59.947771 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb724b1-8776-4466-a6d8-2f0a2d06f9eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:59.949706 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb724b1-8776-4466-a6d8-2f0a2d06f9eb admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:16:59.971555 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1383/2739] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:16:59 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:16:59.975533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149729a-9a3a-4b58-b820-bb9bd2d259cc admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:16:59.986385 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149729a-9a3a-4b58-b820-bb9bd2d259cc admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:16:59.991069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04457ac-3ac9-4ada-aa84-2d3b6e8b2ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:16:59.991726 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04457ac-3ac9-4ada-aa84-2d3b6e8b2ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:16:59.992007 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04457ac-3ac9-4ada-aa84-2d3b6e8b2ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:16:59.992283 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04457ac-3ac9-4ada-aa84-2d3b6e8b2ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:16:59.993037 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04457ac-3ac9-4ada-aa84-2d3b6e8b2ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:16:59.994630 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04457ac-3ac9-4ada-aa84-2d3b6e8b2ff0 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:00.005142 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1357/2740] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:00.009354 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb724b1-8776-4466-a6d8-2f0a2d06f9eb admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:00.019277 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb724b1-8776-4466-a6d8-2f0a2d06f9eb admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:00.024002 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb449f-11d8-4eb6-b22b-70205f927d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:00.024911 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb449f-11d8-4eb6-b22b-70205f927d10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:00.025195 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb449f-11d8-4eb6-b22b-70205f927d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:00.025469 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb449f-11d8-4eb6-b22b-70205f927d10 admin admin] PATH_INFO: `/v3/projects/14386c0adfaa4cf7812891a5b226d78d/users/5e4a89a1db2f4246b8b4d7092ce9856b/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:00.040870 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb449f-11d8-4eb6-b22b-70205f927d10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14386c0adfaa4cf7812891a5b226d78d, user_id=5e4a89a1db2f4246b8b4d7092ce9856b, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:00.043625 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb449f-11d8-4eb6-b22b-70205f927d10 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:00.057555 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1384/2741] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:17:00 2022] PUT /identity/v3/projects/14386c0adfaa4cf7812891a5b226d78d/users/5e4a89a1db2f4246b8b4d7092ce9856b/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:00.061600 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04457ac-3ac9-4ada-aa84-2d3b6e8b2ff0 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:00.071992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04457ac-3ac9-4ada-aa84-2d3b6e8b2ff0 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:00.131824 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8914d30-bf03-46bd-8145-19589d718a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:00.132788 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8914d30-bf03-46bd-8145-19589d718a26 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:00.133243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8914d30-bf03-46bd-8145-19589d718a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:00.133608 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8914d30-bf03-46bd-8145-19589d718a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:00.134352 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8914d30-bf03-46bd-8145-19589d718a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:00.136271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8914d30-bf03-46bd-8145-19589d718a26 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:00.138612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:00.149705 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1358/2742] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:00.153985 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb449f-11d8-4eb6-b22b-70205f927d10 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:00.160071 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:00.166360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb449f-11d8-4eb6-b22b-70205f927d10 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:00.171381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db5f2c-fe8b-409d-974e-74c9e20aee4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:00.172260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db5f2c-fe8b-409d-974e-74c9e20aee4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:00.172544 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db5f2c-fe8b-409d-974e-74c9e20aee4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:00.172874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db5f2c-fe8b-409d-974e-74c9e20aee4f admin admin] PATH_INFO: `/v3/projects/14386c0adfaa4cf7812891a5b226d78d/users/5e4a89a1db2f4246b8b4d7092ce9856b/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:00.175392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db5f2c-fe8b-409d-974e-74c9e20aee4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=14386c0adfaa4cf7812891a5b226d78d, user_id=5e4a89a1db2f4246b8b4d7092ce9856b, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:00.178052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db5f2c-fe8b-409d-974e-74c9e20aee4f admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:00.188394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1385/2743] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:17:00 2022] PUT /identity/v3/projects/14386c0adfaa4cf7812891a5b226d78d/users/5e4a89a1db2f4246b8b4d7092ce9856b/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:00.198662 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78550b6-5faa-44ec-b4f5-04dafecf4c60 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:00.198960 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78550b6-5faa-44ec-b4f5-04dafecf4c60 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:00.199242 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78550b6-5faa-44ec-b4f5-04dafecf4c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:00.226553 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-b78550b6-5faa-44ec-b4f5-04dafecf4c60 None None] MFA Rules not processed for user `5e4a89a1db2f4246b8b4d7092ce9856b`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:00.227769 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-b78550b6-5faa-44ec-b4f5-04dafecf4c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:00.294704 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1359/2744] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:00 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:00.305515 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db5f2c-fe8b-409d-974e-74c9e20aee4f admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:00.307425 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-12db5f2c-fe8b-409d-974e-74c9e20aee4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:00.366427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db5f2c-fe8b-409d-974e-74c9e20aee4f admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:00.370247 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7b506-fbda-4aeb-a990-6a424eb1bd86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:00.370893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7b506-fbda-4aeb-a990-6a424eb1bd86 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:00.371179 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7b506-fbda-4aeb-a990-6a424eb1bd86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:00.371460 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7b506-fbda-4aeb-a990-6a424eb1bd86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:00.380565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa7b506-fbda-4aeb-a990-6a424eb1bd86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:00.382095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa7b506-fbda-4aeb-a990-6a424eb1bd86 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:00.391430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1386/2745] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:17:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:17:01.721256 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78550b6-5faa-44ec-b4f5-04dafecf4c60 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:01.804088 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78550b6-5faa-44ec-b4f5-04dafecf4c60 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:01.808740 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe96e4c-61ad-441f-9b5b-fdeac36b576e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:01.809642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe96e4c-61ad-441f-9b5b-fdeac36b576e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:01.809931 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe96e4c-61ad-441f-9b5b-fdeac36b576e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:01.810214 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe96e4c-61ad-441f-9b5b-fdeac36b576e admin admin] PATH_INFO: `/v3/users/93a9db7f69854e7e8fd6366d5c8534c3` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:01.811428 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe96e4c-61ad-441f-9b5b-fdeac36b576e admin admin] RBAC: Authorizing `identity:delete_user(user_id=93a9db7f69854e7e8fd6366d5c8534c3)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:01.813394 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe96e4c-61ad-441f-9b5b-fdeac36b576e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:01.860635 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ffe96e4c-61ad-441f-9b5b-fdeac36b576e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93a9db7f69854e7e8fd6366d5c8534c3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:01.868347 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ffe96e4c-61ad-441f-9b5b-fdeac36b576e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93a9db7f69854e7e8fd6366d5c8534c3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:01.881156 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ffe96e4c-61ad-441f-9b5b-fdeac36b576e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93a9db7f69854e7e8fd6366d5c8534c3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:01.881828 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ffe96e4c-61ad-441f-9b5b-fdeac36b576e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93a9db7f69854e7e8fd6366d5c8534c3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:01.882444 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-ffe96e4c-61ad-441f-9b5b-fdeac36b576e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93a9db7f69854e7e8fd6366d5c8534c3'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:01.899822 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1360/2746] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:17:01 2022] DELETE /identity/v3/users/93a9db7f69854e7e8fd6366d5c8534c3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:01.904465 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903efd2-38e1-44f3-8bbe-acecfa23e4d4 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:01.905007 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903efd2-38e1-44f3-8bbe-acecfa23e4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:01.905118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903efd2-38e1-44f3-8bbe-acecfa23e4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:01.922866 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-8903efd2-38e1-44f3-8bbe-acecfa23e4d4 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:01.924779 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8903efd2-38e1-44f3-8bbe-acecfa23e4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:01.987873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1387/2747] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:01.997474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe96e4c-61ad-441f-9b5b-fdeac36b576e admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:01.998888 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ffe96e4c-61ad-441f-9b5b-fdeac36b576e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:02.004394 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:02.059293 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe96e4c-61ad-441f-9b5b-fdeac36b576e admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:02.062937 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91daff3a-4b7f-45b9-bfe6-a95d1bf6a195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:02.063577 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91daff3a-4b7f-45b9-bfe6-a95d1bf6a195 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:02.063858 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91daff3a-4b7f-45b9-bfe6-a95d1bf6a195 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:02.064174 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91daff3a-4b7f-45b9-bfe6-a95d1bf6a195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:02.073715 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91daff3a-4b7f-45b9-bfe6-a95d1bf6a195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:02.075168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91daff3a-4b7f-45b9-bfe6-a95d1bf6a195 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:02.084650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1361/2748] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:17:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:17:02.353769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903efd2-38e1-44f3-8bbe-acecfa23e4d4 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:02.355235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8903efd2-38e1-44f3-8bbe-acecfa23e4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:02.370692 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903efd2-38e1-44f3-8bbe-acecfa23e4d4 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:02.376018 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3fa381-694d-4195-b517-c3d7d7426462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:02.377739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fa381-694d-4195-b517-c3d7d7426462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:02.378135 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fa381-694d-4195-b517-c3d7d7426462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:02.378493 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fa381-694d-4195-b517-c3d7d7426462 admin admin] PATH_INFO: `/v3/projects/d790e0a18cd84cc9815b9ef572835004` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:02.379968 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3fa381-694d-4195-b517-c3d7d7426462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d790e0a18cd84cc9815b9ef572835004)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:02.381944 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3fa381-694d-4195-b517-c3d7d7426462 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:02.443376 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3d3fa381-694d-4195-b517-c3d7d7426462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd790e0a18cd84cc9815b9ef572835004', 'request_id': 'req-3d3fa381-694d-4195-b517-c3d7d7426462'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:02.450470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3d3fa381-694d-4195-b517-c3d7d7426462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd790e0a18cd84cc9815b9ef572835004', 'request_id': 'req-3d3fa381-694d-4195-b517-c3d7d7426462'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:02.461161 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3d3fa381-694d-4195-b517-c3d7d7426462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd790e0a18cd84cc9815b9ef572835004', 'request_id': 'req-3d3fa381-694d-4195-b517-c3d7d7426462'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:02.474780 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3d3fa381-694d-4195-b517-c3d7d7426462 admin admin] The token cache is being invalidate because project d790e0a18cd84cc9815b9ef572835004 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:17:02.475149 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3d3fa381-694d-4195-b517-c3d7d7426462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:02.475747 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3d3fa381-694d-4195-b517-c3d7d7426462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:02.477707 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1388/2749] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:17:02 2022] DELETE /identity/v3/projects/d790e0a18cd84cc9815b9ef572835004 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:02.481598 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91daff3a-4b7f-45b9-bfe6-a95d1bf6a195 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:02.492759 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91daff3a-4b7f-45b9-bfe6-a95d1bf6a195 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:02.552015 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb346438-21ec-4252-a665-b55c59ee1651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:02.553045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb346438-21ec-4252-a665-b55c59ee1651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:02.553351 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb346438-21ec-4252-a665-b55c59ee1651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:02.553632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb346438-21ec-4252-a665-b55c59ee1651 admin admin] PATH_INFO: `/v3/users/5e4a89a1db2f4246b8b4d7092ce9856b` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:02.554831 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb346438-21ec-4252-a665-b55c59ee1651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e4a89a1db2f4246b8b4d7092ce9856b)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:02.556781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb346438-21ec-4252-a665-b55c59ee1651 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:02.603570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-eb346438-21ec-4252-a665-b55c59ee1651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e4a89a1db2f4246b8b4d7092ce9856b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:02.610353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-eb346438-21ec-4252-a665-b55c59ee1651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e4a89a1db2f4246b8b4d7092ce9856b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:02.618228 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-eb346438-21ec-4252-a665-b55c59ee1651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e4a89a1db2f4246b8b4d7092ce9856b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:02.618876 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-eb346438-21ec-4252-a665-b55c59ee1651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e4a89a1db2f4246b8b4d7092ce9856b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:02.619629 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-eb346438-21ec-4252-a665-b55c59ee1651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e4a89a1db2f4246b8b4d7092ce9856b'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:02.637128 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1362/2750] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:17:02 2022] DELETE /identity/v3/users/5e4a89a1db2f4246b8b4d7092ce9856b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:02.914854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3fa381-694d-4195-b517-c3d7d7426462 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:02.916310 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-3d3fa381-694d-4195-b517-c3d7d7426462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:02.931416 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3fa381-694d-4195-b517-c3d7d7426462 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:02.945069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:02.994904 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e665ebd-64fb-43e4-8dcb-8026176908aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:02.995692 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e665ebd-64fb-43e4-8dcb-8026176908aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:02.995984 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e665ebd-64fb-43e4-8dcb-8026176908aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:02.996264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e665ebd-64fb-43e4-8dcb-8026176908aa admin admin] PATH_INFO: `/v3/projects/14386c0adfaa4cf7812891a5b226d78d` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:02.997476 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e665ebd-64fb-43e4-8dcb-8026176908aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=14386c0adfaa4cf7812891a5b226d78d)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:02.999394 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e665ebd-64fb-43e4-8dcb-8026176908aa admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:03.045655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3e665ebd-64fb-43e4-8dcb-8026176908aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14386c0adfaa4cf7812891a5b226d78d', 'request_id': 'req-3e665ebd-64fb-43e4-8dcb-8026176908aa'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:03.051900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3e665ebd-64fb-43e4-8dcb-8026176908aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14386c0adfaa4cf7812891a5b226d78d', 'request_id': 'req-3e665ebd-64fb-43e4-8dcb-8026176908aa'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:03.060168 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3e665ebd-64fb-43e4-8dcb-8026176908aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14386c0adfaa4cf7812891a5b226d78d', 'request_id': 'req-3e665ebd-64fb-43e4-8dcb-8026176908aa'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:03.071524 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3e665ebd-64fb-43e4-8dcb-8026176908aa admin admin] The token cache is being invalidate because project 14386c0adfaa4cf7812891a5b226d78d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:17:03.071816 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3e665ebd-64fb-43e4-8dcb-8026176908aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:03.072484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-3e665ebd-64fb-43e4-8dcb-8026176908aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:03.074044 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1389/2751] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:17:02 2022] DELETE /identity/v3/projects/14386c0adfaa4cf7812891a5b226d78d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:17:03.078887 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82ad94-141a-4bdd-ab1d-b6273214a324 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:03.079369 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82ad94-141a-4bdd-ab1d-b6273214a324 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:03.079469 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82ad94-141a-4bdd-ab1d-b6273214a324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:03.098397 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-bd82ad94-141a-4bdd-ab1d-b6273214a324 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:03.099615 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-bd82ad94-141a-4bdd-ab1d-b6273214a324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:03.118274 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:03.162398 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1363/2752] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:03.170125 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588eaaa5-3dfd-49a2-8c16-0e0ef18f1862 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:03.170417 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588eaaa5-3dfd-49a2-8c16-0e0ef18f1862 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:03.170699 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588eaaa5-3dfd-49a2-8c16-0e0ef18f1862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:03.190244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-588eaaa5-3dfd-49a2-8c16-0e0ef18f1862 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:03.191462 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-588eaaa5-3dfd-49a2-8c16-0e0ef18f1862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:03.199738 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1390/2753] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:03.203933 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82ad94-141a-4bdd-ab1d-b6273214a324 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:03.214200 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82ad94-141a-4bdd-ab1d-b6273214a324 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:03.218908 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11197f8-c12e-46ff-990a-c62cec3f330e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:03.219569 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11197f8-c12e-46ff-990a-c62cec3f330e admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:03.219856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11197f8-c12e-46ff-990a-c62cec3f330e admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:03.220152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11197f8-c12e-46ff-990a-c62cec3f330e admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:03.221670 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11197f8-c12e-46ff-990a-c62cec3f330e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:17:03.221983 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11197f8-c12e-46ff-990a-c62cec3f330e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:03.223608 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11197f8-c12e-46ff-990a-c62cec3f330e admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:03.235467 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1364/2754] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:17:03.239733 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588eaaa5-3dfd-49a2-8c16-0e0ef18f1862 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:03.249142 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588eaaa5-3dfd-49a2-8c16-0e0ef18f1862 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:03.253808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a072a-7beb-428d-997d-e6e292ece702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:03.254456 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a072a-7beb-428d-997d-e6e292ece702 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:03.254740 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a072a-7beb-428d-997d-e6e292ece702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:03.255020 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a072a-7beb-428d-997d-e6e292ece702 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:03.255719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a072a-7beb-428d-997d-e6e292ece702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:03.257719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a072a-7beb-428d-997d-e6e292ece702 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:03.283702 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1391/2755] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:17:03 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:03.287781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11197f8-c12e-46ff-990a-c62cec3f330e admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:03.297659 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11197f8-c12e-46ff-990a-c62cec3f330e admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:03.352820 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdbe24-9d71-4e76-bc46-1b4ed0d23d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:03.353480 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdbe24-9d71-4e76-bc46-1b4ed0d23d46 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:03.353761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdbe24-9d71-4e76-bc46-1b4ed0d23d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:03.354032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdbe24-9d71-4e76-bc46-1b4ed0d23d46 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:03.354712 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdbe24-9d71-4e76-bc46-1b4ed0d23d46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:03.356614 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdbe24-9d71-4e76-bc46-1b4ed0d23d46 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:03.380878 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1365/2756] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:17:03 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:03.382141 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a072a-7beb-428d-997d-e6e292ece702 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:03.393321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a072a-7beb-428d-997d-e6e292ece702 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:03.398195 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491ada1-5a2b-4a2a-99e2-0d283799a410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:03.398866 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491ada1-5a2b-4a2a-99e2-0d283799a410 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:03.399147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491ada1-5a2b-4a2a-99e2-0d283799a410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:03.399471 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491ada1-5a2b-4a2a-99e2-0d283799a410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:03.400253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491ada1-5a2b-4a2a-99e2-0d283799a410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:03.401897 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491ada1-5a2b-4a2a-99e2-0d283799a410 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:03.413778 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1392/2757] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:03.417807 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdbe24-9d71-4e76-bc46-1b4ed0d23d46 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:03.428175 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdbe24-9d71-4e76-bc46-1b4ed0d23d46 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:03.432861 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864905bb-37b5-4dbe-a8e7-1ab76930bd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:03.433565 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864905bb-37b5-4dbe-a8e7-1ab76930bd96 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:03.433846 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864905bb-37b5-4dbe-a8e7-1ab76930bd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:03.434128 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864905bb-37b5-4dbe-a8e7-1ab76930bd96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:03.434852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864905bb-37b5-4dbe-a8e7-1ab76930bd96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:03.436487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864905bb-37b5-4dbe-a8e7-1ab76930bd96 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:03.447373 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1366/2758] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:03.451700 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491ada1-5a2b-4a2a-99e2-0d283799a410 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:03.463885 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491ada1-5a2b-4a2a-99e2-0d283799a410 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:03.468574 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9c2fb-a7e3-4374-a60d-3a374424f01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:03.469487 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9c2fb-a7e3-4374-a60d-3a374424f01f admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:03.469767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9c2fb-a7e3-4374-a60d-3a374424f01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:03.470042 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9c2fb-a7e3-4374-a60d-3a374424f01f admin admin] PATH_INFO: `/v3/projects/529550b1de6e49aeae6a9ecd4093308c/users/3f9fd6b196bc4b5d88391798383ad49a/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:03.486989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9c2fb-a7e3-4374-a60d-3a374424f01f admin admin] RBAC: Authorizing `identity:create_grant(project_id=529550b1de6e49aeae6a9ecd4093308c, user_id=3f9fd6b196bc4b5d88391798383ad49a, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:03.489665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9c2fb-a7e3-4374-a60d-3a374424f01f admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:03.503054 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1393/2759] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:17:03 2022] PUT /identity/v3/projects/529550b1de6e49aeae6a9ecd4093308c/users/3f9fd6b196bc4b5d88391798383ad49a/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:03.513950 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a220f-983d-4086-b75d-360303c80517 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:03.514245 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a220f-983d-4086-b75d-360303c80517 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:03.514525 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a220f-983d-4086-b75d-360303c80517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:03.543391 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-244a220f-983d-4086-b75d-360303c80517 None None] MFA Rules not processed for user `3f9fd6b196bc4b5d88391798383ad49a`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:03.544625 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-244a220f-983d-4086-b75d-360303c80517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:03.610116 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1367/2760] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:03.614790 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9c2fb-a7e3-4374-a60d-3a374424f01f admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:03.659232 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:03.678155 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9c2fb-a7e3-4374-a60d-3a374424f01f admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:03.682876 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2d527-7fd6-470a-bcf3-2d6b111302cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:03.683526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2d527-7fd6-470a-bcf3-2d6b111302cb admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:03.683819 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2d527-7fd6-470a-bcf3-2d6b111302cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:03.684101 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2d527-7fd6-470a-bcf3-2d6b111302cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:03.684820 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2d527-7fd6-470a-bcf3-2d6b111302cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:03.686727 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2d527-7fd6-470a-bcf3-2d6b111302cb admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:03.713347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1394/2761] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:17:03 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:03.717464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a220f-983d-4086-b75d-360303c80517 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:03.728102 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a220f-983d-4086-b75d-360303c80517 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:03.744045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:03.794283 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d2d92-7a31-400b-9961-caeb9c948169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:03.795058 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d2d92-7a31-400b-9961-caeb9c948169 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:03.795397 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d2d92-7a31-400b-9961-caeb9c948169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:03.795698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d2d92-7a31-400b-9961-caeb9c948169 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:03.796645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d2d92-7a31-400b-9961-caeb9c948169 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:03.799118 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d2d92-7a31-400b-9961-caeb9c948169 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:03.820231 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1368/2762] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:17:03 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:03.824080 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2d527-7fd6-470a-bcf3-2d6b111302cb admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:03.836265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2d527-7fd6-470a-bcf3-2d6b111302cb admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:03.841063 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fea9be-a315-499e-aab2-bd5f34895f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:03.841725 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fea9be-a315-499e-aab2-bd5f34895f63 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:03.842027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fea9be-a315-499e-aab2-bd5f34895f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:03.842286 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fea9be-a315-499e-aab2-bd5f34895f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:03.843012 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fea9be-a315-499e-aab2-bd5f34895f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:03.844611 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fea9be-a315-499e-aab2-bd5f34895f63 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:03.855244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1395/2763] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:03.859398 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d2d92-7a31-400b-9961-caeb9c948169 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:03.870212 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d2d92-7a31-400b-9961-caeb9c948169 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:03.874924 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb385d6a-6839-4f42-8b2e-fae6e22c4360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:03.875812 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb385d6a-6839-4f42-8b2e-fae6e22c4360 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:03.876092 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb385d6a-6839-4f42-8b2e-fae6e22c4360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:03.876373 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb385d6a-6839-4f42-8b2e-fae6e22c4360 admin admin] PATH_INFO: `/v3/projects/d5e998f297d04f4ab0bdb29379f8f28b/users/25fef89606974e19b4164a1d7c2a9264/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:03.892547 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb385d6a-6839-4f42-8b2e-fae6e22c4360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e998f297d04f4ab0bdb29379f8f28b, user_id=25fef89606974e19b4164a1d7c2a9264, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:03.895306 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb385d6a-6839-4f42-8b2e-fae6e22c4360 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:03.908869 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1369/2764] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:17:03 2022] PUT /identity/v3/projects/d5e998f297d04f4ab0bdb29379f8f28b/users/25fef89606974e19b4164a1d7c2a9264/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:03.912930 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fea9be-a315-499e-aab2-bd5f34895f63 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:03.923216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fea9be-a315-499e-aab2-bd5f34895f63 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:03.980377 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040fa3c-9bbe-4454-9b14-fb34480b43a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:03.981084 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040fa3c-9bbe-4454-9b14-fb34480b43a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:03.981374 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040fa3c-9bbe-4454-9b14-fb34480b43a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:03.981655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040fa3c-9bbe-4454-9b14-fb34480b43a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:03.982384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8040fa3c-9bbe-4454-9b14-fb34480b43a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:03.983986 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8040fa3c-9bbe-4454-9b14-fb34480b43a5 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:03.994401 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1396/2765] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:03.998579 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb385d6a-6839-4f42-8b2e-fae6e22c4360 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:04.009619 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb385d6a-6839-4f42-8b2e-fae6e22c4360 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:04.014243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7c07a-01dd-4ba5-9006-3d3db19327d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:04.015156 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7c07a-01dd-4ba5-9006-3d3db19327d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:04.015403 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7c07a-01dd-4ba5-9006-3d3db19327d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:04.015748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7c07a-01dd-4ba5-9006-3d3db19327d7 admin admin] PATH_INFO: `/v3/projects/d5e998f297d04f4ab0bdb29379f8f28b/users/25fef89606974e19b4164a1d7c2a9264/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:04.017761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a7c07a-01dd-4ba5-9006-3d3db19327d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e998f297d04f4ab0bdb29379f8f28b, user_id=25fef89606974e19b4164a1d7c2a9264, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:04.020428 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a7c07a-01dd-4ba5-9006-3d3db19327d7 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:04.030743 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1370/2766] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:17:03 2022] PUT /identity/v3/projects/d5e998f297d04f4ab0bdb29379f8f28b/users/25fef89606974e19b4164a1d7c2a9264/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:17:04.041117 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc23db0-724b-424c-b241-234896787bb0 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:04.041434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc23db0-724b-424c-b241-234896787bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:04.041719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc23db0-724b-424c-b241-234896787bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:04.069401 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-fcc23db0-724b-424c-b241-234896787bb0 None None] MFA Rules not processed for user `25fef89606974e19b4164a1d7c2a9264`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:04.070535 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-fcc23db0-724b-424c-b241-234896787bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:04.138058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1397/2767] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:04.148957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7c07a-01dd-4ba5-9006-3d3db19327d7 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:04.150839 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-11a7c07a-01dd-4ba5-9006-3d3db19327d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:04.209561 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7c07a-01dd-4ba5-9006-3d3db19327d7 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:04.213203 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829368b-1181-45fc-8460-26c830110da6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:04.213847 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829368b-1181-45fc-8460-26c830110da6 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:04.214129 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829368b-1181-45fc-8460-26c830110da6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:04.214408 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829368b-1181-45fc-8460-26c830110da6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:04.271317 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829368b-1181-45fc-8460-26c830110da6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:04.272777 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829368b-1181-45fc-8460-26c830110da6 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:04.281620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1371/2768] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:17:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:17:04.945118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc23db0-724b-424c-b241-234896787bb0 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:04.946707 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-fcc23db0-724b-424c-b241-234896787bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:04.993200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:05.010665 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc23db0-724b-424c-b241-234896787bb0 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:05.013867 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc6a45-87cf-4c48-9080-6a97f9352f11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:05.014522 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc6a45-87cf-4c48-9080-6a97f9352f11 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:05.014808 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc6a45-87cf-4c48-9080-6a97f9352f11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:05.015087 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc6a45-87cf-4c48-9080-6a97f9352f11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:05.016933 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-e8bc6a45-87cf-4c48-9080-6a97f9352f11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:05.075618 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc6a45-87cf-4c48-9080-6a97f9352f11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:05.077084 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc6a45-87cf-4c48-9080-6a97f9352f11 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:05.087499 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1398/2769] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:17:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:17:09.944215 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829368b-1181-45fc-8460-26c830110da6 service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:09.945671 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6829368b-1181-45fc-8460-26c830110da6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:09.958069 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829368b-1181-45fc-8460-26c830110da6 service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:09.961843 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdcd07-2570-4c5d-8750-44909be751f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:09.962502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdcd07-2570-4c5d-8750-44909be751f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:09.963430 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdcd07-2570-4c5d-8750-44909be751f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:09.963718 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdcd07-2570-4c5d-8750-44909be751f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:09.973757 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdcd07-2570-4c5d-8750-44909be751f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:09.975215 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdcd07-2570-4c5d-8750-44909be751f3 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:09.985137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1372/2770] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:17:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:17:15.082610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=31 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:19.980677 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:INSERT=5 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:27.319989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc6a45-87cf-4c48-9080-6a97f9352f11 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:27.328854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc6a45-87cf-4c48-9080-6a97f9352f11 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:27.333031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf08677-1983-44a3-b3d0-a630be0c687d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:27.333669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf08677-1983-44a3-b3d0-a630be0c687d service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:27.333955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf08677-1983-44a3-b3d0-a630be0c687d service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:27.334234 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf08677-1983-44a3-b3d0-a630be0c687d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:27.335982 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ecf08677-1983-44a3-b3d0-a630be0c687d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:27.399155 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf08677-1983-44a3-b3d0-a630be0c687d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:27.400601 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf08677-1983-44a3-b3d0-a630be0c687d service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:27.411530 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1399/2771] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:17:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:17:30.867010 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdcd07-2570-4c5d-8750-44909be751f3 service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:30.879646 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdcd07-2570-4c5d-8750-44909be751f3 service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:30.885211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074baea-3051-4581-984e-661bad392f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:30.886197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074baea-3051-4581-984e-661bad392f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:30.886638 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074baea-3051-4581-984e-661bad392f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:30.887005 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074baea-3051-4581-984e-661bad392f6c admin admin] PATH_INFO: `/v3/users/3f9fd6b196bc4b5d88391798383ad49a` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:30.888326 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074baea-3051-4581-984e-661bad392f6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f9fd6b196bc4b5d88391798383ad49a)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:30.890875 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074baea-3051-4581-984e-661bad392f6c admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:30.945487 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8074baea-3051-4581-984e-661bad392f6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f9fd6b196bc4b5d88391798383ad49a'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:30.955088 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8074baea-3051-4581-984e-661bad392f6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f9fd6b196bc4b5d88391798383ad49a'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:30.967897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8074baea-3051-4581-984e-661bad392f6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f9fd6b196bc4b5d88391798383ad49a'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:30.968528 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8074baea-3051-4581-984e-661bad392f6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f9fd6b196bc4b5d88391798383ad49a'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:30.969191 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8074baea-3051-4581-984e-661bad392f6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f9fd6b196bc4b5d88391798383ad49a'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:30.988456 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1373/2772] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:17:30 2022] DELETE /identity/v3/users/3f9fd6b196bc4b5d88391798383ad49a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:30.993107 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e33a74-cc63-4651-bcf0-a8069a31753d None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:30.993636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e33a74-cc63-4651-bcf0-a8069a31753d None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:30.993954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e33a74-cc63-4651-bcf0-a8069a31753d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:31.011746 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-40e33a74-cc63-4651-bcf0-a8069a31753d None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:31.013259 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-40e33a74-cc63-4651-bcf0-a8069a31753d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:31.075463 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1400/2773] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:31.085693 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074baea-3051-4581-984e-661bad392f6c admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:31.087160 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8074baea-3051-4581-984e-661bad392f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:31.145794 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074baea-3051-4581-984e-661bad392f6c admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:31.149507 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63284e-bad9-43ad-87de-6c9ef140784c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:31.150156 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63284e-bad9-43ad-87de-6c9ef140784c service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:31.150441 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63284e-bad9-43ad-87de-6c9ef140784c service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:31.150718 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63284e-bad9-43ad-87de-6c9ef140784c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:31.161019 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63284e-bad9-43ad-87de-6c9ef140784c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:31.162482 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63284e-bad9-43ad-87de-6c9ef140784c service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:31.172453 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1374/2774] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:17:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:17:31.461482 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e33a74-cc63-4651-bcf0-a8069a31753d None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:31.463300 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-40e33a74-cc63-4651-bcf0-a8069a31753d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:31.482468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e33a74-cc63-4651-bcf0-a8069a31753d None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:31.488524 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda7346-d179-4975-b818-833bb93757ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:31.489440 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda7346-d179-4975-b818-833bb93757ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:31.490069 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda7346-d179-4975-b818-833bb93757ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:31.490287 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda7346-d179-4975-b818-833bb93757ee admin admin] PATH_INFO: `/v3/projects/529550b1de6e49aeae6a9ecd4093308c` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:31.491636 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda7346-d179-4975-b818-833bb93757ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=529550b1de6e49aeae6a9ecd4093308c)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:31.494067 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda7346-d179-4975-b818-833bb93757ee admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:31.545682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2bda7346-d179-4975-b818-833bb93757ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '529550b1de6e49aeae6a9ecd4093308c', 'request_id': 'req-2bda7346-d179-4975-b818-833bb93757ee'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:31.554188 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2bda7346-d179-4975-b818-833bb93757ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '529550b1de6e49aeae6a9ecd4093308c', 'request_id': 'req-2bda7346-d179-4975-b818-833bb93757ee'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:31.562550 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2bda7346-d179-4975-b818-833bb93757ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '529550b1de6e49aeae6a9ecd4093308c', 'request_id': 'req-2bda7346-d179-4975-b818-833bb93757ee'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:31.574829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2bda7346-d179-4975-b818-833bb93757ee admin admin] The token cache is being invalidate because project 529550b1de6e49aeae6a9ecd4093308c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:17:31.575132 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2bda7346-d179-4975-b818-833bb93757ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:31.575814 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-2bda7346-d179-4975-b818-833bb93757ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:31.577632 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1401/2775] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:17:31 2022] DELETE /identity/v3/projects/529550b1de6e49aeae6a9ecd4093308c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:31.581989 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63284e-bad9-43ad-87de-6c9ef140784c service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:31.592863 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63284e-bad9-43ad-87de-6c9ef140784c service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:31.664428 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806be57f-3eb9-4b13-bd1e-31e0d490b1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:31.667145 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806be57f-3eb9-4b13-bd1e-31e0d490b1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:31.667543 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806be57f-3eb9-4b13-bd1e-31e0d490b1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:31.667929 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806be57f-3eb9-4b13-bd1e-31e0d490b1cf admin admin] PATH_INFO: `/v3/users/25fef89606974e19b4164a1d7c2a9264` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:31.669454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806be57f-3eb9-4b13-bd1e-31e0d490b1cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=25fef89606974e19b4164a1d7c2a9264)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:31.671549 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806be57f-3eb9-4b13-bd1e-31e0d490b1cf admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:31.694342 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:31.736656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-806be57f-3eb9-4b13-bd1e-31e0d490b1cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25fef89606974e19b4164a1d7c2a9264'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:31.746135 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-806be57f-3eb9-4b13-bd1e-31e0d490b1cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25fef89606974e19b4164a1d7c2a9264'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:31.756339 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-806be57f-3eb9-4b13-bd1e-31e0d490b1cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25fef89606974e19b4164a1d7c2a9264'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:31.757025 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-806be57f-3eb9-4b13-bd1e-31e0d490b1cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25fef89606974e19b4164a1d7c2a9264'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:31.757633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-806be57f-3eb9-4b13-bd1e-31e0d490b1cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25fef89606974e19b4164a1d7c2a9264'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:31.786230 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1375/2776] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:17:31 2022] DELETE /identity/v3/users/25fef89606974e19b4164a1d7c2a9264 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:32.534806 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda7346-d179-4975-b818-833bb93757ee admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:32.536398 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2bda7346-d179-4975-b818-833bb93757ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:32.558267 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda7346-d179-4975-b818-833bb93757ee admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:32.577710 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:32.632924 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8e131-cb02-4256-b272-f3060352efd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:32.633864 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8e131-cb02-4256-b272-f3060352efd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:32.634159 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8e131-cb02-4256-b272-f3060352efd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:32.634434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8e131-cb02-4256-b272-f3060352efd0 admin admin] PATH_INFO: `/v3/projects/d5e998f297d04f4ab0bdb29379f8f28b` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:32.635852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8e131-cb02-4256-b272-f3060352efd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5e998f297d04f4ab0bdb29379f8f28b)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:32.637812 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8e131-cb02-4256-b272-f3060352efd0 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:32.694052 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c9c8e131-cb02-4256-b272-f3060352efd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5e998f297d04f4ab0bdb29379f8f28b', 'request_id': 'req-c9c8e131-cb02-4256-b272-f3060352efd0'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:32.700507 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c9c8e131-cb02-4256-b272-f3060352efd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5e998f297d04f4ab0bdb29379f8f28b', 'request_id': 'req-c9c8e131-cb02-4256-b272-f3060352efd0'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:32.711209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c9c8e131-cb02-4256-b272-f3060352efd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5e998f297d04f4ab0bdb29379f8f28b', 'request_id': 'req-c9c8e131-cb02-4256-b272-f3060352efd0'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:32.723936 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c9c8e131-cb02-4256-b272-f3060352efd0 admin admin] The token cache is being invalidate because project d5e998f297d04f4ab0bdb29379f8f28b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:17:32.724296 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c9c8e131-cb02-4256-b272-f3060352efd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:32.724908 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-c9c8e131-cb02-4256-b272-f3060352efd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:32.726720 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1402/2777] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:17:32 2022] DELETE /identity/v3/projects/d5e998f297d04f4ab0bdb29379f8f28b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:32.731999 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f296d-3179-48ba-883e-fed679148717 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:32.732492 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f296d-3179-48ba-883e-fed679148717 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:32.732621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f296d-3179-48ba-883e-fed679148717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:32.751951 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-1a5f296d-3179-48ba-883e-fed679148717 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:32.753742 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1a5f296d-3179-48ba-883e-fed679148717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:32.826860 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1376/2778] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:32.890564 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d25b6-2838-4ee2-9ff4-4a6f76e4a430 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:32.890856 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d25b6-2838-4ee2-9ff4-4a6f76e4a430 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:32.891143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d25b6-2838-4ee2-9ff4-4a6f76e4a430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:32.919013 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-af5d25b6-2838-4ee2-9ff4-4a6f76e4a430 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:32.920281 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-af5d25b6-2838-4ee2-9ff4-4a6f76e4a430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:32.932647 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1403/2779] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:32.934144 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f296d-3179-48ba-883e-fed679148717 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:32.945916 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f296d-3179-48ba-883e-fed679148717 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:32.951190 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305aa2b-a0da-4daf-a052-09bf6e2d43a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:32.951945 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305aa2b-a0da-4daf-a052-09bf6e2d43a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:32.952290 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305aa2b-a0da-4daf-a052-09bf6e2d43a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:32.952639 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305aa2b-a0da-4daf-a052-09bf6e2d43a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:32.953473 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305aa2b-a0da-4daf-a052-09bf6e2d43a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:17:32.953868 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305aa2b-a0da-4daf-a052-09bf6e2d43a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:32.955617 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305aa2b-a0da-4daf-a052-09bf6e2d43a9 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:32.973485 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1377/2780] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 22:17:32.977940 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d25b6-2838-4ee2-9ff4-4a6f76e4a430 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:32.988434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d25b6-2838-4ee2-9ff4-4a6f76e4a430 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:32.993385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49f3c7-61b0-47db-9d66-5e81e035e7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:32.994118 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49f3c7-61b0-47db-9d66-5e81e035e7f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:32.994414 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49f3c7-61b0-47db-9d66-5e81e035e7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:32.994694 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49f3c7-61b0-47db-9d66-5e81e035e7f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:32.995408 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49f3c7-61b0-47db-9d66-5e81e035e7f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:32.997356 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49f3c7-61b0-47db-9d66-5e81e035e7f1 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:33.025597 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1404/2781] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:17:32 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:33.029870 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305aa2b-a0da-4daf-a052-09bf6e2d43a9 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:33.041472 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305aa2b-a0da-4daf-a052-09bf6e2d43a9 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:33.102516 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ff623-0a55-442e-a20f-5504172b3349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:33.103261 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ff623-0a55-442e-a20f-5504172b3349 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:33.103556 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ff623-0a55-442e-a20f-5504172b3349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:33.103864 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ff623-0a55-442e-a20f-5504172b3349 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:33.104534 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ff623-0a55-442e-a20f-5504172b3349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:33.106534 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ff623-0a55-442e-a20f-5504172b3349 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:33.127217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1378/2782] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:17:33 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:33.131701 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49f3c7-61b0-47db-9d66-5e81e035e7f1 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:33.142534 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49f3c7-61b0-47db-9d66-5e81e035e7f1 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:33.148073 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b065c79-891e-4ac1-95d9-2b9d46366359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:33.148751 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b065c79-891e-4ac1-95d9-2b9d46366359 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:33.149048 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b065c79-891e-4ac1-95d9-2b9d46366359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:33.149329 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b065c79-891e-4ac1-95d9-2b9d46366359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:33.150079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b065c79-891e-4ac1-95d9-2b9d46366359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:33.151674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b065c79-891e-4ac1-95d9-2b9d46366359 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:33.163384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1405/2783] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:33.167994 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ff623-0a55-442e-a20f-5504172b3349 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:33.179197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ff623-0a55-442e-a20f-5504172b3349 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:33.184298 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3f29c-3e82-43fe-82be-a397335ebf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:33.185017 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3f29c-3e82-43fe-82be-a397335ebf73 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:33.185459 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3f29c-3e82-43fe-82be-a397335ebf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:33.185748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3f29c-3e82-43fe-82be-a397335ebf73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:33.186495 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3f29c-3e82-43fe-82be-a397335ebf73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:33.188153 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3f29c-3e82-43fe-82be-a397335ebf73 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:33.195009 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:33.209099 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1379/2784] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:33.209582 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b065c79-891e-4ac1-95d9-2b9d46366359 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:33.220056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b065c79-891e-4ac1-95d9-2b9d46366359 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:33.226189 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93562938-def2-4a41-b1c0-9e5e83a5da98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:33.228137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93562938-def2-4a41-b1c0-9e5e83a5da98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:33.228137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93562938-def2-4a41-b1c0-9e5e83a5da98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:33.228137 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93562938-def2-4a41-b1c0-9e5e83a5da98 admin admin] PATH_INFO: `/v3/projects/b5afdd6cdeb347c3afc42bcb07e87c16/users/0438f7a11a234d1abf1f74aa2f714a0d/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:33.245131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93562938-def2-4a41-b1c0-9e5e83a5da98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5afdd6cdeb347c3afc42bcb07e87c16, user_id=0438f7a11a234d1abf1f74aa2f714a0d, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:33.247961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93562938-def2-4a41-b1c0-9e5e83a5da98 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:33.271079 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1406/2785] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:17:33 2022] PUT /identity/v3/projects/b5afdd6cdeb347c3afc42bcb07e87c16/users/0438f7a11a234d1abf1f74aa2f714a0d/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:33.286850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85478ae-578b-42ec-bb12-5e83888fbf9e None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:33.287138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85478ae-578b-42ec-bb12-5e83888fbf9e None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:33.287419 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85478ae-578b-42ec-bb12-5e83888fbf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:33.317246 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-a85478ae-578b-42ec-bb12-5e83888fbf9e None None] MFA Rules not processed for user `0438f7a11a234d1abf1f74aa2f714a0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:33.318748 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a85478ae-578b-42ec-bb12-5e83888fbf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:33.400782 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1380/2786] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:17:33.402050 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93562938-def2-4a41-b1c0-9e5e83a5da98 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:33.455363 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:33.477960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93562938-def2-4a41-b1c0-9e5e83a5da98 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:33.483043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912843fb-75a7-4cdc-bc01-a3b75dfc4ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:33.483719 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912843fb-75a7-4cdc-bc01-a3b75dfc4ccd admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:33.484131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912843fb-75a7-4cdc-bc01-a3b75dfc4ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:33.484434 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912843fb-75a7-4cdc-bc01-a3b75dfc4ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:33.485186 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912843fb-75a7-4cdc-bc01-a3b75dfc4ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:33.487233 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912843fb-75a7-4cdc-bc01-a3b75dfc4ccd admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:33.518769 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1407/2787] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:17:33 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:33.523060 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85478ae-578b-42ec-bb12-5e83888fbf9e None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:33.534150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85478ae-578b-42ec-bb12-5e83888fbf9e None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:33.594474 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf975718-de29-4081-8dcb-de29d357cde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:33.595166 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf975718-de29-4081-8dcb-de29d357cde4 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:33.595452 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf975718-de29-4081-8dcb-de29d357cde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:33.595735 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf975718-de29-4081-8dcb-de29d357cde4 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:33.596546 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf975718-de29-4081-8dcb-de29d357cde4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:33.598836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf975718-de29-4081-8dcb-de29d357cde4 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:33.634632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1381/2788] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:17:33 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:33.635572 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912843fb-75a7-4cdc-bc01-a3b75dfc4ccd admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:33.649323 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912843fb-75a7-4cdc-bc01-a3b75dfc4ccd admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:33.661954 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642de15a-d44e-47de-bc2f-b70f28b7f5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:33.662693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642de15a-d44e-47de-bc2f-b70f28b7f5b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:33.662989 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642de15a-d44e-47de-bc2f-b70f28b7f5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:33.663327 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642de15a-d44e-47de-bc2f-b70f28b7f5b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:33.664171 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642de15a-d44e-47de-bc2f-b70f28b7f5b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:33.665948 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642de15a-d44e-47de-bc2f-b70f28b7f5b5 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:33.679449 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1408/2789] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:33.683775 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf975718-de29-4081-8dcb-de29d357cde4 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:33.695739 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf975718-de29-4081-8dcb-de29d357cde4 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:33.701596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23c7a6-3c5b-4e1b-a8f7-b7d325a1a192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:33.711302 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23c7a6-3c5b-4e1b-a8f7-b7d325a1a192 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:33.711800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23c7a6-3c5b-4e1b-a8f7-b7d325a1a192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:33.713207 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23c7a6-3c5b-4e1b-a8f7-b7d325a1a192 admin admin] PATH_INFO: `/v3/projects/c99a2e60f4be4d209e31a15cdf9cfc31/users/f7e00f6793ea4be580b6d0e12673d638/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:33.735750 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23c7a6-3c5b-4e1b-a8f7-b7d325a1a192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c99a2e60f4be4d209e31a15cdf9cfc31, user_id=f7e00f6793ea4be580b6d0e12673d638, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:33.740528 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23c7a6-3c5b-4e1b-a8f7-b7d325a1a192 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:33.763022 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1382/2790] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:17:33 2022] PUT /identity/v3/projects/c99a2e60f4be4d209e31a15cdf9cfc31/users/f7e00f6793ea4be580b6d0e12673d638/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:33.769130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642de15a-d44e-47de-bc2f-b70f28b7f5b5 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:33.780921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642de15a-d44e-47de-bc2f-b70f28b7f5b5 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:33.859379 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239b1ca-9aa2-4af2-bf20-46e8d6f6b041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:33.860057 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239b1ca-9aa2-4af2-bf20-46e8d6f6b041 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:33.860339 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239b1ca-9aa2-4af2-bf20-46e8d6f6b041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:33.860614 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239b1ca-9aa2-4af2-bf20-46e8d6f6b041 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:33.861376 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239b1ca-9aa2-4af2-bf20-46e8d6f6b041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:33.863702 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239b1ca-9aa2-4af2-bf20-46e8d6f6b041 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:33.878384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1409/2791] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 22:17:33.882281 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23c7a6-3c5b-4e1b-a8f7-b7d325a1a192 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:33.894384 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23c7a6-3c5b-4e1b-a8f7-b7d325a1a192 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:33.900658 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a352221-fd81-4f1b-b953-d46f30e3aad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:33.902055 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a352221-fd81-4f1b-b953-d46f30e3aad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:33.902296 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a352221-fd81-4f1b-b953-d46f30e3aad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:33.902643 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a352221-fd81-4f1b-b953-d46f30e3aad4 admin admin] PATH_INFO: `/v3/projects/c99a2e60f4be4d209e31a15cdf9cfc31/users/f7e00f6793ea4be580b6d0e12673d638/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:33.904985 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a352221-fd81-4f1b-b953-d46f30e3aad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c99a2e60f4be4d209e31a15cdf9cfc31, user_id=f7e00f6793ea4be580b6d0e12673d638, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:33.908169 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a352221-fd81-4f1b-b953-d46f30e3aad4 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:33.921024 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1383/2792] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:17:33 2022] PUT /identity/v3/projects/c99a2e60f4be4d209e31a15cdf9cfc31/users/f7e00f6793ea4be580b6d0e12673d638/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:33.932119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409763b-ef46-46e4-8214-33b6a5377044 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:33.932418 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409763b-ef46-46e4-8214-33b6a5377044 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:33.932730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409763b-ef46-46e4-8214-33b6a5377044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:33.978893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-0409763b-ef46-46e4-8214-33b6a5377044 None None] MFA Rules not processed for user `f7e00f6793ea4be580b6d0e12673d638`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:33.980002 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0409763b-ef46-46e4-8214-33b6a5377044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:33.988226 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a352221-fd81-4f1b-b953-d46f30e3aad4 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:33.990229 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-9a352221-fd81-4f1b-b953-d46f30e3aad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:34.046063 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:34.065870 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1410/2793] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:34.073831 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a352221-fd81-4f1b-b953-d46f30e3aad4 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:34.077266 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409763b-ef46-46e4-8214-33b6a5377044 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:34.078640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0409763b-ef46-46e4-8214-33b6a5377044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:34.078792 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c95563-f7e9-466b-a45b-b33e0c92f3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:34.079679 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c95563-f7e9-466b-a45b-b33e0c92f3ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:34.079965 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c95563-f7e9-466b-a45b-b33e0c92f3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:34.080243 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c95563-f7e9-466b-a45b-b33e0c92f3ba admin admin] PATH_INFO: `/v3/users/24a41c09f2e4487f9897c9888a0d752f` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:34.081775 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c95563-f7e9-466b-a45b-b33e0c92f3ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=24a41c09f2e4487f9897c9888a0d752f)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:34.084326 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c95563-f7e9-466b-a45b-b33e0c92f3ba admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:34.125390 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:34.135353 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-02c95563-f7e9-466b-a45b-b33e0c92f3ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24a41c09f2e4487f9897c9888a0d752f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.142816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-02c95563-f7e9-466b-a45b-b33e0c92f3ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24a41c09f2e4487f9897c9888a0d752f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.149830 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409763b-ef46-46e4-8214-33b6a5377044 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:34.150844 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-02c95563-f7e9-466b-a45b-b33e0c92f3ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24a41c09f2e4487f9897c9888a0d752f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.151680 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-02c95563-f7e9-466b-a45b-b33e0c92f3ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24a41c09f2e4487f9897c9888a0d752f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.152566 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-02c95563-f7e9-466b-a45b-b33e0c92f3ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24a41c09f2e4487f9897c9888a0d752f'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.153962 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9fe0f-4f17-45ce-8743-279bb397dce4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:34.154616 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9fe0f-4f17-45ce-8743-279bb397dce4 service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:34.154905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9fe0f-4f17-45ce-8743-279bb397dce4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:34.155187 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9fe0f-4f17-45ce-8743-279bb397dce4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:34.157027 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-82a9fe0f-4f17-45ce-8743-279bb397dce4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:34.169661 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a9fe0f-4f17-45ce-8743-279bb397dce4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:34.171156 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a9fe0f-4f17-45ce-8743-279bb397dce4 service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:34.171371 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1384/2794] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:17:33 2022] DELETE /identity/v3/users/24a41c09f2e4487f9897c9888a0d752f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:34.181961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1411/2795] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:17:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:17:34.211507 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c95563-f7e9-466b-a45b-b33e0c92f3ba admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:34.213249 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-02c95563-f7e9-466b-a45b-b33e0c92f3ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:34.292070 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c95563-f7e9-466b-a45b-b33e0c92f3ba admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:34.298168 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc29a97-3b46-4fa6-ba0f-429bdd4380ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:34.299126 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc29a97-3b46-4fa6-ba0f-429bdd4380ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:34.300029 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc29a97-3b46-4fa6-ba0f-429bdd4380ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:34.300510 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc29a97-3b46-4fa6-ba0f-429bdd4380ec admin admin] PATH_INFO: `/v3/users/0438f7a11a234d1abf1f74aa2f714a0d` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:34.302125 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc29a97-3b46-4fa6-ba0f-429bdd4380ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=0438f7a11a234d1abf1f74aa2f714a0d)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:34.304781 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc29a97-3b46-4fa6-ba0f-429bdd4380ec admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:34.357422 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8fc29a97-3b46-4fa6-ba0f-429bdd4380ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0438f7a11a234d1abf1f74aa2f714a0d'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.365179 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8fc29a97-3b46-4fa6-ba0f-429bdd4380ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0438f7a11a234d1abf1f74aa2f714a0d'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.372839 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8fc29a97-3b46-4fa6-ba0f-429bdd4380ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0438f7a11a234d1abf1f74aa2f714a0d'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.373434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8fc29a97-3b46-4fa6-ba0f-429bdd4380ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0438f7a11a234d1abf1f74aa2f714a0d'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.374061 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-8fc29a97-3b46-4fa6-ba0f-429bdd4380ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0438f7a11a234d1abf1f74aa2f714a0d'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.375633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:34.376660 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9fe0f-4f17-45ce-8743-279bb397dce4 service cinder] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:34.378533 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-82a9fe0f-4f17-45ce-8743-279bb397dce4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:34.404429 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9fe0f-4f17-45ce-8743-279bb397dce4 service cinder] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:34.407434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1385/2796] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:17:34 2022] DELETE /identity/v3/users/0438f7a11a234d1abf1f74aa2f714a0d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:34.412506 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e97373-8f9b-4ab7-b99f-6cba673b5d22 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:34.412683 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c507f0-3665-4beb-bbb3-a569aa01a805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:34.412833 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e97373-8f9b-4ab7-b99f-6cba673b5d22 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:34.413124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e97373-8f9b-4ab7-b99f-6cba673b5d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:34.413439 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c507f0-3665-4beb-bbb3-a569aa01a805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:34.413717 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c507f0-3665-4beb-bbb3-a569aa01a805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:34.414006 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c507f0-3665-4beb-bbb3-a569aa01a805 admin admin] PATH_INFO: `/v3/projects/80e867d86756459f9e4f53a4238d393d` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:34.415280 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c507f0-3665-4beb-bbb3-a569aa01a805 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80e867d86756459f9e4f53a4238d393d)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:34.417283 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c507f0-3665-4beb-bbb3-a569aa01a805 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:34.434520 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-82e97373-8f9b-4ab7-b99f-6cba673b5d22 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:34.435632 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-82e97373-8f9b-4ab7-b99f-6cba673b5d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:34.486307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-58c507f0-3665-4beb-bbb3-a569aa01a805 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80e867d86756459f9e4f53a4238d393d', 'request_id': 'req-58c507f0-3665-4beb-bbb3-a569aa01a805'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.492238 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-58c507f0-3665-4beb-bbb3-a569aa01a805 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80e867d86756459f9e4f53a4238d393d', 'request_id': 'req-58c507f0-3665-4beb-bbb3-a569aa01a805'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.501500 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1386/2797] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:34.502111 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-58c507f0-3665-4beb-bbb3-a569aa01a805 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80e867d86756459f9e4f53a4238d393d', 'request_id': 'req-58c507f0-3665-4beb-bbb3-a569aa01a805'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.511944 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e97373-8f9b-4ab7-b99f-6cba673b5d22 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:34.513852 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-82e97373-8f9b-4ab7-b99f-6cba673b5d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:34.521801 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-58c507f0-3665-4beb-bbb3-a569aa01a805 admin admin] The token cache is being invalidate because project 80e867d86756459f9e4f53a4238d393d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:17:34.522015 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-58c507f0-3665-4beb-bbb3-a569aa01a805 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.522613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-58c507f0-3665-4beb-bbb3-a569aa01a805 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.524464 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1412/2798] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:17:34 2022] DELETE /identity/v3/projects/80e867d86756459f9e4f53a4238d393d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:17:34.528303 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c507f0-3665-4beb-bbb3-a569aa01a805 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:34.529674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-58c507f0-3665-4beb-bbb3-a569aa01a805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:34.586272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e97373-8f9b-4ab7-b99f-6cba673b5d22 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:34.604760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c507f0-3665-4beb-bbb3-a569aa01a805 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:34.610820 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c4d2a-7315-4276-9cbe-ef7db045456f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:34.612877 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c4d2a-7315-4276-9cbe-ef7db045456f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:34.613367 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c4d2a-7315-4276-9cbe-ef7db045456f admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:34.613840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c4d2a-7315-4276-9cbe-ef7db045456f admin admin] PATH_INFO: `/v3/users/8bd938cbe36b4f0db2d3253a291963bb` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:34.615727 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c4d2a-7315-4276-9cbe-ef7db045456f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bd938cbe36b4f0db2d3253a291963bb)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:34.617839 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c4d2a-7315-4276-9cbe-ef7db045456f admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:34.644981 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47cfef-4b7b-49b4-8ceb-dadfaae33c32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:34.645715 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47cfef-4b7b-49b4-8ceb-dadfaae33c32 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:34.646004 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47cfef-4b7b-49b4-8ceb-dadfaae33c32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:34.646284 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47cfef-4b7b-49b4-8ceb-dadfaae33c32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:34.649117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-5f47cfef-4b7b-49b4-8ceb-dadfaae33c32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:34.662358 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47cfef-4b7b-49b4-8ceb-dadfaae33c32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:34.663796 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47cfef-4b7b-49b4-8ceb-dadfaae33c32 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:34.666404 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-351c4d2a-7315-4276-9cbe-ef7db045456f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bd938cbe36b4f0db2d3253a291963bb'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.673726 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1387/2799] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:17:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 22:17:34.679953 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-351c4d2a-7315-4276-9cbe-ef7db045456f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bd938cbe36b4f0db2d3253a291963bb'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.681999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:34.688763 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-351c4d2a-7315-4276-9cbe-ef7db045456f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bd938cbe36b4f0db2d3253a291963bb'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.698761 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-351c4d2a-7315-4276-9cbe-ef7db045456f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bd938cbe36b4f0db2d3253a291963bb'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.699679 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-351c4d2a-7315-4276-9cbe-ef7db045456f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bd938cbe36b4f0db2d3253a291963bb'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:34.706149 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1413/2800] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:17:34 2022] DELETE /identity/v3/users/8bd938cbe36b4f0db2d3253a291963bb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:34.898314 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47cfef-4b7b-49b4-8ceb-dadfaae33c32 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:34.909387 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47cfef-4b7b-49b4-8ceb-dadfaae33c32 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:34.958744 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:34.984476 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c4d2a-7315-4276-9cbe-ef7db045456f admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:34.985983 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-351c4d2a-7315-4276-9cbe-ef7db045456f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:34.987940 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab54a08-4747-4a9d-bf26-309e929a2bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:34.989078 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab54a08-4747-4a9d-bf26-309e929a2bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:34.989463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab54a08-4747-4a9d-bf26-309e929a2bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:34.990090 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab54a08-4747-4a9d-bf26-309e929a2bc7 admin admin] PATH_INFO: `/v3/projects/840b8dfce15e451294dc402954ee5c6d` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:34.991942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab54a08-4747-4a9d-bf26-309e929a2bc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=840b8dfce15e451294dc402954ee5c6d)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:34.999290 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab54a08-4747-4a9d-bf26-309e929a2bc7 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:35.002974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c4d2a-7315-4276-9cbe-ef7db045456f admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:35.008915 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e7452-f842-4ccf-b7af-7a017107edcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:35.010232 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e7452-f842-4ccf-b7af-7a017107edcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:35.010514 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e7452-f842-4ccf-b7af-7a017107edcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:35.010819 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e7452-f842-4ccf-b7af-7a017107edcb admin admin] PATH_INFO: `/v3/projects/b5afdd6cdeb347c3afc42bcb07e87c16` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:35.012874 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e7452-f842-4ccf-b7af-7a017107edcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5afdd6cdeb347c3afc42bcb07e87c16)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:35.015241 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e7452-f842-4ccf-b7af-7a017107edcb admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:35.065347 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1ab54a08-4747-4a9d-bf26-309e929a2bc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '840b8dfce15e451294dc402954ee5c6d', 'request_id': 'req-1ab54a08-4747-4a9d-bf26-309e929a2bc7'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.072108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1ab54a08-4747-4a9d-bf26-309e929a2bc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '840b8dfce15e451294dc402954ee5c6d', 'request_id': 'req-1ab54a08-4747-4a9d-bf26-309e929a2bc7'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.075260 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-5f8e7452-f842-4ccf-b7af-7a017107edcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5afdd6cdeb347c3afc42bcb07e87c16', 'request_id': 'req-5f8e7452-f842-4ccf-b7af-7a017107edcb'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.080565 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1ab54a08-4747-4a9d-bf26-309e929a2bc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '840b8dfce15e451294dc402954ee5c6d', 'request_id': 'req-1ab54a08-4747-4a9d-bf26-309e929a2bc7'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.081533 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-5f8e7452-f842-4ccf-b7af-7a017107edcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5afdd6cdeb347c3afc42bcb07e87c16', 'request_id': 'req-5f8e7452-f842-4ccf-b7af-7a017107edcb'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.091072 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-5f8e7452-f842-4ccf-b7af-7a017107edcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5afdd6cdeb347c3afc42bcb07e87c16', 'request_id': 'req-5f8e7452-f842-4ccf-b7af-7a017107edcb'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.093396 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1ab54a08-4747-4a9d-bf26-309e929a2bc7 admin admin] The token cache is being invalidate because project 840b8dfce15e451294dc402954ee5c6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:17:35.093695 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1ab54a08-4747-4a9d-bf26-309e929a2bc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.094418 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-1ab54a08-4747-4a9d-bf26-309e929a2bc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.096670 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1388/2801] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:17:34 2022] DELETE /identity/v3/projects/840b8dfce15e451294dc402954ee5c6d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:35.104875 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-5f8e7452-f842-4ccf-b7af-7a017107edcb admin admin] The token cache is being invalidate because project b5afdd6cdeb347c3afc42bcb07e87c16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:17:35.105174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-5f8e7452-f842-4ccf-b7af-7a017107edcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.105877 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-5f8e7452-f842-4ccf-b7af-7a017107edcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.107489 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1414/2802] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:17:34 2022] DELETE /identity/v3/projects/b5afdd6cdeb347c3afc42bcb07e87c16 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:17:35.111585 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab54a08-4747-4a9d-bf26-309e929a2bc7 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:35.113741 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1ab54a08-4747-4a9d-bf26-309e929a2bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:35.183933 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab54a08-4747-4a9d-bf26-309e929a2bc7 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:35.245977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58969119-8407-48f9-ab48-06b0c217e175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:35.246882 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58969119-8407-48f9-ab48-06b0c217e175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:35.247167 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58969119-8407-48f9-ab48-06b0c217e175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:35.247444 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58969119-8407-48f9-ab48-06b0c217e175 admin admin] PATH_INFO: `/v3/users/f7e00f6793ea4be580b6d0e12673d638` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:35.248778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58969119-8407-48f9-ab48-06b0c217e175 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7e00f6793ea4be580b6d0e12673d638)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:35.250804 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58969119-8407-48f9-ab48-06b0c217e175 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:35.273905 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:35.305765 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-58969119-8407-48f9-ab48-06b0c217e175 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7e00f6793ea4be580b6d0e12673d638'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.312939 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-58969119-8407-48f9-ab48-06b0c217e175 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7e00f6793ea4be580b6d0e12673d638'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.322889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-58969119-8407-48f9-ab48-06b0c217e175 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7e00f6793ea4be580b6d0e12673d638'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.323534 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-58969119-8407-48f9-ab48-06b0c217e175 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7e00f6793ea4be580b6d0e12673d638'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.324380 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-58969119-8407-48f9-ab48-06b0c217e175 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7e00f6793ea4be580b6d0e12673d638'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.344494 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1389/2803] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:17:35 2022] DELETE /identity/v3/users/f7e00f6793ea4be580b6d0e12673d638 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:35.616742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e7452-f842-4ccf-b7af-7a017107edcb admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:35.618278 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5f8e7452-f842-4ccf-b7af-7a017107edcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:35.632016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e7452-f842-4ccf-b7af-7a017107edcb admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:35.693129 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605122e-1a5c-4711-873d-b232dfbe1edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:35.693922 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605122e-1a5c-4711-873d-b232dfbe1edd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:35.694212 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605122e-1a5c-4711-873d-b232dfbe1edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:35.694482 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605122e-1a5c-4711-873d-b232dfbe1edd admin admin] PATH_INFO: `/v3/projects/c99a2e60f4be4d209e31a15cdf9cfc31` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:35.695682 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d605122e-1a5c-4711-873d-b232dfbe1edd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c99a2e60f4be4d209e31a15cdf9cfc31)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:35.697629 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d605122e-1a5c-4711-873d-b232dfbe1edd admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:35.743902 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d605122e-1a5c-4711-873d-b232dfbe1edd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c99a2e60f4be4d209e31a15cdf9cfc31', 'request_id': 'req-d605122e-1a5c-4711-873d-b232dfbe1edd'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.749589 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d605122e-1a5c-4711-873d-b232dfbe1edd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c99a2e60f4be4d209e31a15cdf9cfc31', 'request_id': 'req-d605122e-1a5c-4711-873d-b232dfbe1edd'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.757890 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d605122e-1a5c-4711-873d-b232dfbe1edd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c99a2e60f4be4d209e31a15cdf9cfc31', 'request_id': 'req-d605122e-1a5c-4711-873d-b232dfbe1edd'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.764924 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:35.770716 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d605122e-1a5c-4711-873d-b232dfbe1edd admin admin] The token cache is being invalidate because project c99a2e60f4be4d209e31a15cdf9cfc31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:17:35.771187 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d605122e-1a5c-4711-873d-b232dfbe1edd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.772342 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-d605122e-1a5c-4711-873d-b232dfbe1edd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:35.774209 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1415/2804] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:17:35 2022] DELETE /identity/v3/projects/c99a2e60f4be4d209e31a15cdf9cfc31 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:35.779272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e67da2-e779-458d-b4b2-819a8b7f8569 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:35.779560 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e67da2-e779-458d-b4b2-819a8b7f8569 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:35.779846 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e67da2-e779-458d-b4b2-819a8b7f8569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:35.797881 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-52e67da2-e779-458d-b4b2-819a8b7f8569 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:35.799093 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-52e67da2-e779-458d-b4b2-819a8b7f8569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:35.866456 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1390/2805] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:35.875572 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9b04e-0b09-4249-b06b-ed3e9ce33e84 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:35.875861 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9b04e-0b09-4249-b06b-ed3e9ce33e84 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:35.876143 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9b04e-0b09-4249-b06b-ed3e9ce33e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:35.895686 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-8bd9b04e-0b09-4249-b06b-ed3e9ce33e84 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:35.896973 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-8bd9b04e-0b09-4249-b06b-ed3e9ce33e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:35.905034 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1416/2806] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:35.909666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e67da2-e779-458d-b4b2-819a8b7f8569 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:35.919645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e67da2-e779-458d-b4b2-819a8b7f8569 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:35.924345 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860611e-8233-4999-9308-24698d4168e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:35.925032 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860611e-8233-4999-9308-24698d4168e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:35.925397 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860611e-8233-4999-9308-24698d4168e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:35.925684 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860611e-8233-4999-9308-24698d4168e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:35.929897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860611e-8233-4999-9308-24698d4168e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:17:35.930236 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860611e-8233-4999-9308-24698d4168e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:35.932828 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860611e-8233-4999-9308-24698d4168e6 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:35.948528 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1391/2807] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:17:35.953093 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9b04e-0b09-4249-b06b-ed3e9ce33e84 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:35.965043 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9b04e-0b09-4249-b06b-ed3e9ce33e84 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:35.969964 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da75b0-6b1a-4160-992a-821e6a7bbf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:35.970781 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da75b0-6b1a-4160-992a-821e6a7bbf18 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:35.971205 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da75b0-6b1a-4160-992a-821e6a7bbf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:35.971705 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da75b0-6b1a-4160-992a-821e6a7bbf18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:35.972427 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da75b0-6b1a-4160-992a-821e6a7bbf18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:35.974791 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da75b0-6b1a-4160-992a-821e6a7bbf18 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:36.004883 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1417/2808] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:17:35 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:36.006405 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860611e-8233-4999-9308-24698d4168e6 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:36.016209 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860611e-8233-4999-9308-24698d4168e6 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:36.072679 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3784d-42e4-4268-944b-f32492d094ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:36.073362 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3784d-42e4-4268-944b-f32492d094ad admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:36.073642 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3784d-42e4-4268-944b-f32492d094ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:36.073915 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3784d-42e4-4268-944b-f32492d094ad admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:36.074593 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b3784d-42e4-4268-944b-f32492d094ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:36.076493 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b3784d-42e4-4268-944b-f32492d094ad admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:36.099548 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1392/2809] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:17:36 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:36.104584 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da75b0-6b1a-4160-992a-821e6a7bbf18 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:36.115533 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da75b0-6b1a-4160-992a-821e6a7bbf18 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:36.120126 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738754e-0822-43d1-89fc-33f55f473526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:36.120804 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738754e-0822-43d1-89fc-33f55f473526 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:36.121088 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738754e-0822-43d1-89fc-33f55f473526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:36.121365 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738754e-0822-43d1-89fc-33f55f473526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:36.122083 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738754e-0822-43d1-89fc-33f55f473526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:36.123668 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738754e-0822-43d1-89fc-33f55f473526 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:36.134697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1418/2810] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:36.138590 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3784d-42e4-4268-944b-f32492d094ad admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:36.148816 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3784d-42e4-4268-944b-f32492d094ad admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:36.153624 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d510bb9-e731-4fc5-82d8-e16da4d36627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:36.154279 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d510bb9-e731-4fc5-82d8-e16da4d36627 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:36.154562 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d510bb9-e731-4fc5-82d8-e16da4d36627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:36.154838 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d510bb9-e731-4fc5-82d8-e16da4d36627 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:36.155557 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d510bb9-e731-4fc5-82d8-e16da4d36627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:36.157206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d510bb9-e731-4fc5-82d8-e16da4d36627 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:36.167125 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1393/2811] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:36.171162 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738754e-0822-43d1-89fc-33f55f473526 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:36.181899 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738754e-0822-43d1-89fc-33f55f473526 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:36.186502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc8b31-f0e7-4be7-9f9d-5127093a27cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:36.187381 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc8b31-f0e7-4be7-9f9d-5127093a27cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:36.187666 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc8b31-f0e7-4be7-9f9d-5127093a27cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:36.187947 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc8b31-f0e7-4be7-9f9d-5127093a27cf admin admin] PATH_INFO: `/v3/projects/3c6a48e23af34b79abdd852a197ba444/users/9f1cfafd7dec41dea0fd4a6eb2b6de87/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:36.203468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc8b31-f0e7-4be7-9f9d-5127093a27cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c6a48e23af34b79abdd852a197ba444, user_id=9f1cfafd7dec41dea0fd4a6eb2b6de87, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:36.206326 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc8b31-f0e7-4be7-9f9d-5127093a27cf admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:36.218996 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1419/2812] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:17:36 2022] PUT /identity/v3/projects/3c6a48e23af34b79abdd852a197ba444/users/9f1cfafd7dec41dea0fd4a6eb2b6de87/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:36.229208 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce2fe6-f1fa-4a25-94ec-7f4ea2e2bb19 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:36.229572 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce2fe6-f1fa-4a25-94ec-7f4ea2e2bb19 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:36.229866 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce2fe6-f1fa-4a25-94ec-7f4ea2e2bb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:36.235836 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:36.266874 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-adce2fe6-f1fa-4a25-94ec-7f4ea2e2bb19 None None] MFA Rules not processed for user `9f1cfafd7dec41dea0fd4a6eb2b6de87`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:36.268151 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-adce2fe6-f1fa-4a25-94ec-7f4ea2e2bb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:36.345583 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1394/2813] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:36.350418 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc8b31-f0e7-4be7-9f9d-5127093a27cf admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:36.417675 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc8b31-f0e7-4be7-9f9d-5127093a27cf admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:36.422638 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114fd3e-52a2-4e37-b5d7-a8f08ed92fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:36.423294 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114fd3e-52a2-4e37-b5d7-a8f08ed92fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:36.423583 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114fd3e-52a2-4e37-b5d7-a8f08ed92fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:36.423865 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114fd3e-52a2-4e37-b5d7-a8f08ed92fc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:36.425437 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114fd3e-52a2-4e37-b5d7-a8f08ed92fc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:36.427439 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114fd3e-52a2-4e37-b5d7-a8f08ed92fc8 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:36.452399 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1420/2814] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:17:36 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:36.456530 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce2fe6-f1fa-4a25-94ec-7f4ea2e2bb19 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:36.467151 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce2fe6-f1fa-4a25-94ec-7f4ea2e2bb19 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:36.523839 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f9862-4eab-46d7-9994-ad78e9336701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:36.524511 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f9862-4eab-46d7-9994-ad78e9336701 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:36.524826 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f9862-4eab-46d7-9994-ad78e9336701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:36.525106 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f9862-4eab-46d7-9994-ad78e9336701 admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:36.525800 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f9862-4eab-46d7-9994-ad78e9336701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:36.527722 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f9862-4eab-46d7-9994-ad78e9336701 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:36.548557 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1395/2815] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:17:36 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 22:17:36.553369 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114fd3e-52a2-4e37-b5d7-a8f08ed92fc8 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:36.563805 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114fd3e-52a2-4e37-b5d7-a8f08ed92fc8 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:36.570371 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25123ea9-c53f-4977-9cd2-266cce0291fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:36.571338 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25123ea9-c53f-4977-9cd2-266cce0291fe admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:36.571767 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25123ea9-c53f-4977-9cd2-266cce0291fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:36.572189 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25123ea9-c53f-4977-9cd2-266cce0291fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:36.573361 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25123ea9-c53f-4977-9cd2-266cce0291fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:36.575897 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25123ea9-c53f-4977-9cd2-266cce0291fe admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:36.590591 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1421/2816] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:36.595615 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f9862-4eab-46d7-9994-ad78e9336701 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:36.607323 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f9862-4eab-46d7-9994-ad78e9336701 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:36.612601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64569e4d-b541-48b4-a683-cfa3eeaf3e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:36.613501 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64569e4d-b541-48b4-a683-cfa3eeaf3e87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:36.613785 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64569e4d-b541-48b4-a683-cfa3eeaf3e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:36.614059 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64569e4d-b541-48b4-a683-cfa3eeaf3e87 admin admin] PATH_INFO: `/v3/projects/d39cb7579b974630a20b296b6ae8054d/users/a808545efa224775a50eea34bd038868/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:36.629192 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64569e4d-b541-48b4-a683-cfa3eeaf3e87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d39cb7579b974630a20b296b6ae8054d, user_id=a808545efa224775a50eea34bd038868, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:36.631847 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64569e4d-b541-48b4-a683-cfa3eeaf3e87 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:36.645253 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1396/2817] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:17:36 2022] PUT /identity/v3/projects/d39cb7579b974630a20b296b6ae8054d/users/a808545efa224775a50eea34bd038868/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:36.649330 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25123ea9-c53f-4977-9cd2-266cce0291fe admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:36.659836 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25123ea9-c53f-4977-9cd2-266cce0291fe admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:36.716566 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbc6cb-4696-4d5c-9f85-7ee5b90cd811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:36.717244 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbc6cb-4696-4d5c-9f85-7ee5b90cd811 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:36.717461 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbc6cb-4696-4d5c-9f85-7ee5b90cd811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:36.717742 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbc6cb-4696-4d5c-9f85-7ee5b90cd811 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:36.718468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbc6cb-4696-4d5c-9f85-7ee5b90cd811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:36.720083 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbc6cb-4696-4d5c-9f85-7ee5b90cd811 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:36.730503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1422/2818] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:36.734621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64569e4d-b541-48b4-a683-cfa3eeaf3e87 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:36.746097 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64569e4d-b541-48b4-a683-cfa3eeaf3e87 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:36.750769 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129952d-aca1-4f7f-9c67-78b12fe306d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:36.751671 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129952d-aca1-4f7f-9c67-78b12fe306d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:36.751955 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129952d-aca1-4f7f-9c67-78b12fe306d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:36.752264 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129952d-aca1-4f7f-9c67-78b12fe306d2 admin admin] PATH_INFO: `/v3/projects/d39cb7579b974630a20b296b6ae8054d/users/a808545efa224775a50eea34bd038868/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:36.754455 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129952d-aca1-4f7f-9c67-78b12fe306d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d39cb7579b974630a20b296b6ae8054d, user_id=a808545efa224775a50eea34bd038868, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:36.757156 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129952d-aca1-4f7f-9c67-78b12fe306d2 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:36.767289 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1397/2819] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:17:36 2022] PUT /identity/v3/projects/d39cb7579b974630a20b296b6ae8054d/users/a808545efa224775a50eea34bd038868/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:36.778269 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53663bf7-bb40-4a29-958d-27097398fa2a None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:36.778561 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53663bf7-bb40-4a29-958d-27097398fa2a None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:36.778844 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53663bf7-bb40-4a29-958d-27097398fa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:36.786109 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:36.809774 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-53663bf7-bb40-4a29-958d-27097398fa2a None None] MFA Rules not processed for user `a808545efa224775a50eea34bd038868`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:36.811049 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-53663bf7-bb40-4a29-958d-27097398fa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:36.879735 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1423/2820] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:36.890270 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129952d-aca1-4f7f-9c67-78b12fe306d2 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:36.892003 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a129952d-aca1-4f7f-9c67-78b12fe306d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:36.941198 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:36.953324 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129952d-aca1-4f7f-9c67-78b12fe306d2 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:36.957490 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572779e-e12e-4fff-9c74-91a201a13bb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:36.958137 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572779e-e12e-4fff-9c74-91a201a13bb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:36.958424 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572779e-e12e-4fff-9c74-91a201a13bb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:36.958705 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572779e-e12e-4fff-9c74-91a201a13bb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:36.968343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572779e-e12e-4fff-9c74-91a201a13bb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:36.969828 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572779e-e12e-4fff-9c74-91a201a13bb6 service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:36.980261 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1398/2821] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:17:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:17:37.077322 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53663bf7-bb40-4a29-958d-27097398fa2a None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:37.089221 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53663bf7-bb40-4a29-958d-27097398fa2a None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:37.093064 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f5cac-5782-46da-a5dd-8091f558ed9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:37.093739 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f5cac-5782-46da-a5dd-8091f558ed9e service cinder] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:37.094028 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f5cac-5782-46da-a5dd-8091f558ed9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:37.094311 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f5cac-5782-46da-a5dd-8091f558ed9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:37.156730 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f5cac-5782-46da-a5dd-8091f558ed9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:37.158188 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f5cac-5782-46da-a5dd-8091f558ed9e service cinder] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:37.167603 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1424/2822] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:17:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:17:39.729183 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572779e-e12e-4fff-9c74-91a201a13bb6 service cinder] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:39.791621 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572779e-e12e-4fff-9c74-91a201a13bb6 service cinder] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:39.796056 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5529dad-fefb-4240-9607-36eccd090891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:39.796942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5529dad-fefb-4240-9607-36eccd090891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:39.797225 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5529dad-fefb-4240-9607-36eccd090891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:39.797496 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5529dad-fefb-4240-9607-36eccd090891 admin admin] PATH_INFO: `/v3/users/9f1cfafd7dec41dea0fd4a6eb2b6de87` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:39.798659 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5529dad-fefb-4240-9607-36eccd090891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f1cfafd7dec41dea0fd4a6eb2b6de87)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:39.801136 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5529dad-fefb-4240-9607-36eccd090891 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:39.849763 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a5529dad-fefb-4240-9607-36eccd090891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f1cfafd7dec41dea0fd4a6eb2b6de87'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:39.856393 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a5529dad-fefb-4240-9607-36eccd090891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f1cfafd7dec41dea0fd4a6eb2b6de87'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:39.864212 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a5529dad-fefb-4240-9607-36eccd090891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f1cfafd7dec41dea0fd4a6eb2b6de87'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:39.864865 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a5529dad-fefb-4240-9607-36eccd090891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f1cfafd7dec41dea0fd4a6eb2b6de87'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:39.865467 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-a5529dad-fefb-4240-9607-36eccd090891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f1cfafd7dec41dea0fd4a6eb2b6de87'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:39.883664 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1399/2823] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:17:39 2022] DELETE /identity/v3/users/9f1cfafd7dec41dea0fd4a6eb2b6de87 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:39.888371 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22410c8c-d5a7-44a3-874f-9db2262cbc74 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:39.888897 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22410c8c-d5a7-44a3-874f-9db2262cbc74 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:39.889021 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22410c8c-d5a7-44a3-874f-9db2262cbc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:39.906364 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-22410c8c-d5a7-44a3-874f-9db2262cbc74 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:39.907980 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-22410c8c-d5a7-44a3-874f-9db2262cbc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:39.968436 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1425/2824] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:39.977612 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5529dad-fefb-4240-9607-36eccd090891 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:39.979021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-a5529dad-fefb-4240-9607-36eccd090891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:40.037732 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5529dad-fefb-4240-9607-36eccd090891 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:40.041426 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43154c9d-8a24-4d66-a536-327a73081bc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:40.042068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43154c9d-8a24-4d66-a536-327a73081bc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:40.042349 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43154c9d-8a24-4d66-a536-327a73081bc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:40.042618 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43154c9d-8a24-4d66-a536-327a73081bc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:40.051760 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43154c9d-8a24-4d66-a536-327a73081bc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:40.053234 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43154c9d-8a24-4d66-a536-327a73081bc4 service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:40.062568 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1400/2825] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:17:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:17:40.345589 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22410c8c-d5a7-44a3-874f-9db2262cbc74 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:40.347029 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-22410c8c-d5a7-44a3-874f-9db2262cbc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:40.361505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22410c8c-d5a7-44a3-874f-9db2262cbc74 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:40.366431 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ec440-29bc-421b-a0e1-a1f07d9dec4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:40.367440 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ec440-29bc-421b-a0e1-a1f07d9dec4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:40.367847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ec440-29bc-421b-a0e1-a1f07d9dec4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:40.368309 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ec440-29bc-421b-a0e1-a1f07d9dec4e admin admin] PATH_INFO: `/v3/projects/3c6a48e23af34b79abdd852a197ba444` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:40.369585 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ec440-29bc-421b-a0e1-a1f07d9dec4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c6a48e23af34b79abdd852a197ba444)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:40.371825 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ec440-29bc-421b-a0e1-a1f07d9dec4e admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:40.399131 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:40.423442 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-601ec440-29bc-421b-a0e1-a1f07d9dec4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c6a48e23af34b79abdd852a197ba444', 'request_id': 'req-601ec440-29bc-421b-a0e1-a1f07d9dec4e'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:40.429468 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-601ec440-29bc-421b-a0e1-a1f07d9dec4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c6a48e23af34b79abdd852a197ba444', 'request_id': 'req-601ec440-29bc-421b-a0e1-a1f07d9dec4e'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:40.436627 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-601ec440-29bc-421b-a0e1-a1f07d9dec4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c6a48e23af34b79abdd852a197ba444', 'request_id': 'req-601ec440-29bc-421b-a0e1-a1f07d9dec4e'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:40.448891 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-601ec440-29bc-421b-a0e1-a1f07d9dec4e admin admin] The token cache is being invalidate because project 3c6a48e23af34b79abdd852a197ba444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:17:40.449190 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-601ec440-29bc-421b-a0e1-a1f07d9dec4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:40.449830 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-601ec440-29bc-421b-a0e1-a1f07d9dec4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:40.451370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1426/2826] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:17:40 2022] DELETE /identity/v3/projects/3c6a48e23af34b79abdd852a197ba444 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:40.455311 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43154c9d-8a24-4d66-a536-327a73081bc4 service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:40.465712 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43154c9d-8a24-4d66-a536-327a73081bc4 service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:40.470406 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:40.526894 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e2195-45bf-4f88-8b21-e4a33be89559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:40.527772 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e2195-45bf-4f88-8b21-e4a33be89559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:40.528062 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e2195-45bf-4f88-8b21-e4a33be89559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:40.528342 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e2195-45bf-4f88-8b21-e4a33be89559 admin admin] PATH_INFO: `/v3/users/a808545efa224775a50eea34bd038868` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:40.529576 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e2195-45bf-4f88-8b21-e4a33be89559 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a808545efa224775a50eea34bd038868)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:40.531533 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e2195-45bf-4f88-8b21-e4a33be89559 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:40.577697 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-da6e2195-45bf-4f88-8b21-e4a33be89559 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a808545efa224775a50eea34bd038868'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:40.584819 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-da6e2195-45bf-4f88-8b21-e4a33be89559 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a808545efa224775a50eea34bd038868'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:40.593175 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-da6e2195-45bf-4f88-8b21-e4a33be89559 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a808545efa224775a50eea34bd038868'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:40.593771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-da6e2195-45bf-4f88-8b21-e4a33be89559 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a808545efa224775a50eea34bd038868'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:40.594377 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-da6e2195-45bf-4f88-8b21-e4a33be89559 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a808545efa224775a50eea34bd038868'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:40.611829 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1401/2827] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:17:40 2022] DELETE /identity/v3/users/a808545efa224775a50eea34bd038868 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:40.889544 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ec440-29bc-421b-a0e1-a1f07d9dec4e admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:40.891046 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-601ec440-29bc-421b-a0e1-a1f07d9dec4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:40.904921 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ec440-29bc-421b-a0e1-a1f07d9dec4e admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:40.964353 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a7cd0-cdd8-44f9-a314-c9aa4147bb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:40.965261 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a7cd0-cdd8-44f9-a314-c9aa4147bb39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:40.965547 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a7cd0-cdd8-44f9-a314-c9aa4147bb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:40.965829 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a7cd0-cdd8-44f9-a314-c9aa4147bb39 admin admin] PATH_INFO: `/v3/projects/d39cb7579b974630a20b296b6ae8054d` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:40.967030 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a7cd0-cdd8-44f9-a314-c9aa4147bb39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d39cb7579b974630a20b296b6ae8054d)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:40.968973 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a7cd0-cdd8-44f9-a314-c9aa4147bb39 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:41.016467 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-be7a7cd0-cdd8-44f9-a314-c9aa4147bb39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd39cb7579b974630a20b296b6ae8054d', 'request_id': 'req-be7a7cd0-cdd8-44f9-a314-c9aa4147bb39'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:41.022002 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-be7a7cd0-cdd8-44f9-a314-c9aa4147bb39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd39cb7579b974630a20b296b6ae8054d', 'request_id': 'req-be7a7cd0-cdd8-44f9-a314-c9aa4147bb39'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:41.030307 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-be7a7cd0-cdd8-44f9-a314-c9aa4147bb39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd39cb7579b974630a20b296b6ae8054d', 'request_id': 'req-be7a7cd0-cdd8-44f9-a314-c9aa4147bb39'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:41.042183 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-be7a7cd0-cdd8-44f9-a314-c9aa4147bb39 admin admin] The token cache is being invalidate because project d39cb7579b974630a20b296b6ae8054d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:17:41.042474 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-be7a7cd0-cdd8-44f9-a314-c9aa4147bb39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:41.043234 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-be7a7cd0-cdd8-44f9-a314-c9aa4147bb39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:41.044802 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1427/2828] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:17:40 2022] DELETE /identity/v3/projects/d39cb7579b974630a20b296b6ae8054d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:41.049729 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea1875-60f0-4eb1-bde9-e8b666bb3c39 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:41.050230 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea1875-60f0-4eb1-bde9-e8b666bb3c39 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:41.051374 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea1875-60f0-4eb1-bde9-e8b666bb3c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:41.069977 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-64ea1875-60f0-4eb1-bde9-e8b666bb3c39 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:41.071197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-64ea1875-60f0-4eb1-bde9-e8b666bb3c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:41.132947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1402/2829] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:41.140905 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2a695-41bd-4c72-a13a-fd552948b198 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:41.141202 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2a695-41bd-4c72-a13a-fd552948b198 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:41.141482 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2a695-41bd-4c72-a13a-fd552948b198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:41.159474 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-fcb2a695-41bd-4c72-a13a-fd552948b198 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:41.161379 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-fcb2a695-41bd-4c72-a13a-fd552948b198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:41.169351 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1428/2830] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:41.173697 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea1875-60f0-4eb1-bde9-e8b666bb3c39 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:41.184423 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea1875-60f0-4eb1-bde9-e8b666bb3c39 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:41.189115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f55d0a-587c-4a0c-a264-cd972351b452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:41.189778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f55d0a-587c-4a0c-a264-cd972351b452 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:41.190061 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f55d0a-587c-4a0c-a264-cd972351b452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:41.190337 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f55d0a-587c-4a0c-a264-cd972351b452 admin admin] PATH_INFO: `/v3/domains` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:41.191279 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f55d0a-587c-4a0c-a264-cd972351b452 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80219) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:17:41.191589 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f55d0a-587c-4a0c-a264-cd972351b452 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:41.193575 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f55d0a-587c-4a0c-a264-cd972351b452 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:41.205089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1403/2831] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:17:41.209634 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2a695-41bd-4c72-a13a-fd552948b198 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:41.219349 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2a695-41bd-4c72-a13a-fd552948b198 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:41.224674 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d2b95-e2dc-44e5-aa3e-ca8b0f230a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:41.225355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d2b95-e2dc-44e5-aa3e-ca8b0f230a7d admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:41.225637 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d2b95-e2dc-44e5-aa3e-ca8b0f230a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:41.225920 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d2b95-e2dc-44e5-aa3e-ca8b0f230a7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:41.226616 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d2b95-e2dc-44e5-aa3e-ca8b0f230a7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:41.228521 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d2b95-e2dc-44e5-aa3e-ca8b0f230a7d admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:41.254622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1429/2832] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:17:41 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:41.258797 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f55d0a-587c-4a0c-a264-cd972351b452 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:41.268906 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f55d0a-587c-4a0c-a264-cd972351b452 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:41.282542 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:41.328842 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44ee9c-f09d-40bb-a1f0-a1542cac3d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:41.329516 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44ee9c-f09d-40bb-a1f0-a1542cac3d8f admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:41.329827 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44ee9c-f09d-40bb-a1f0-a1542cac3d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:41.330100 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44ee9c-f09d-40bb-a1f0-a1542cac3d8f admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:41.330778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44ee9c-f09d-40bb-a1f0-a1542cac3d8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:41.333215 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44ee9c-f09d-40bb-a1f0-a1542cac3d8f admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:41.353376 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1404/2833] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:17:41 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:41.357467 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d2b95-e2dc-44e5-aa3e-ca8b0f230a7d admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:41.367964 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d2b95-e2dc-44e5-aa3e-ca8b0f230a7d admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:41.372607 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9178c5-c621-4ecd-b9aa-d729eaeb7d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:41.373331 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9178c5-c621-4ecd-b9aa-d729eaeb7d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:41.373617 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9178c5-c621-4ecd-b9aa-d729eaeb7d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:41.373897 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9178c5-c621-4ecd-b9aa-d729eaeb7d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:41.374622 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9178c5-c621-4ecd-b9aa-d729eaeb7d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:41.376211 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9178c5-c621-4ecd-b9aa-d729eaeb7d61 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:41.387016 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1430/2834] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:41.391113 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44ee9c-f09d-40bb-a1f0-a1542cac3d8f admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:41.401935 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44ee9c-f09d-40bb-a1f0-a1542cac3d8f admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:41.406620 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1cff0-08ce-4552-b62a-d732ae713279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:41.407346 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1cff0-08ce-4552-b62a-d732ae713279 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:41.407676 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1cff0-08ce-4552-b62a-d732ae713279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:41.407998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1cff0-08ce-4552-b62a-d732ae713279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:41.408854 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1cff0-08ce-4552-b62a-d732ae713279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:41.411162 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1cff0-08ce-4552-b62a-d732ae713279 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:41.421135 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1405/2835] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:41.425597 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9178c5-c621-4ecd-b9aa-d729eaeb7d61 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:41.430924 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:41.437392 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9178c5-c621-4ecd-b9aa-d729eaeb7d61 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:41.443873 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f87b4-0c46-4c64-813e-8b25e3bc7356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:41.444777 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f87b4-0c46-4c64-813e-8b25e3bc7356 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:41.445086 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f87b4-0c46-4c64-813e-8b25e3bc7356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:41.445456 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f87b4-0c46-4c64-813e-8b25e3bc7356 admin admin] PATH_INFO: `/v3/projects/57ea794654fb4411b0fadacb632918a2/users/99f9909fd4c54b0eb47d2d07a15fc117/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:41.460797 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f87b4-0c46-4c64-813e-8b25e3bc7356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57ea794654fb4411b0fadacb632918a2, user_id=99f9909fd4c54b0eb47d2d07a15fc117, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:41.463492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f87b4-0c46-4c64-813e-8b25e3bc7356 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:41.477216 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1431/2836] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:17:41 2022] PUT /identity/v3/projects/57ea794654fb4411b0fadacb632918a2/users/99f9909fd4c54b0eb47d2d07a15fc117/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:41.487633 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe1ad5-7c3d-435e-a981-7b0f8cb1d359 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:41.488018 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe1ad5-7c3d-435e-a981-7b0f8cb1d359 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:41.488666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe1ad5-7c3d-435e-a981-7b0f8cb1d359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:41.516222 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-d2fe1ad5-7c3d-435e-a981-7b0f8cb1d359 None None] MFA Rules not processed for user `99f9909fd4c54b0eb47d2d07a15fc117`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:41.517491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-d2fe1ad5-7c3d-435e-a981-7b0f8cb1d359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:41.582107 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1406/2837] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:41.586693 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f87b4-0c46-4c64-813e-8b25e3bc7356 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:41.648656 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f87b4-0c46-4c64-813e-8b25e3bc7356 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:41.653544 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff451e-b6e1-4712-b320-22f1ba37ac4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:41.654194 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff451e-b6e1-4712-b320-22f1ba37ac4a admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:41.654479 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff451e-b6e1-4712-b320-22f1ba37ac4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:41.654760 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff451e-b6e1-4712-b320-22f1ba37ac4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:41.655452 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff451e-b6e1-4712-b320-22f1ba37ac4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:41.657382 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff451e-b6e1-4712-b320-22f1ba37ac4a admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:41.682968 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1432/2838] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:17:41 2022] POST /identity/v3/projects => generated 389 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:41.687075 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe1ad5-7c3d-435e-a981-7b0f8cb1d359 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:41.699355 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe1ad5-7c3d-435e-a981-7b0f8cb1d359 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:41.759192 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7ed86-6782-4c97-9d5b-528e8978428a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:41.759927 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7ed86-6782-4c97-9d5b-528e8978428a admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:41.760152 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7ed86-6782-4c97-9d5b-528e8978428a admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:41.760433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7ed86-6782-4c97-9d5b-528e8978428a admin admin] PATH_INFO: `/v3/users` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:41.761153 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa7ed86-6782-4c97-9d5b-528e8978428a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:41.763085 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa7ed86-6782-4c97-9d5b-528e8978428a admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:41.784838 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1407/2839] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:17:41 2022] POST /identity/v3/users => generated 301 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:41.788617 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff451e-b6e1-4712-b320-22f1ba37ac4a admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:41.799422 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff451e-b6e1-4712-b320-22f1ba37ac4a admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:41.804613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d15b86-98a4-4b66-ac86-10eb48a437dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:41.805497 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d15b86-98a4-4b66-ac86-10eb48a437dc admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:41.805633 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d15b86-98a4-4b66-ac86-10eb48a437dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:41.805907 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d15b86-98a4-4b66-ac86-10eb48a437dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:41.806606 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d15b86-98a4-4b66-ac86-10eb48a437dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:41.808220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d15b86-98a4-4b66-ac86-10eb48a437dc admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:41.823809 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1433/2840] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:41.824974 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7ed86-6782-4c97-9d5b-528e8978428a admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:41.836211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7ed86-6782-4c97-9d5b-528e8978428a admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:41.840999 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fda82d-04c5-4cdc-941a-f84e51118c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:41.841879 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fda82d-04c5-4cdc-941a-f84e51118c32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:41.842158 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fda82d-04c5-4cdc-941a-f84e51118c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:41.842429 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fda82d-04c5-4cdc-941a-f84e51118c32 admin admin] PATH_INFO: `/v3/projects/cf808d4541ca49eb914d7f31a78d6ef7/users/b7609bb5dc474e05b63d6566cc4c3251/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:41.855769 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:41.860547 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fda82d-04c5-4cdc-941a-f84e51118c32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf808d4541ca49eb914d7f31a78d6ef7, user_id=b7609bb5dc474e05b63d6566cc4c3251, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:41.863946 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fda82d-04c5-4cdc-941a-f84e51118c32 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:41.885138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1408/2841] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:17:41 2022] PUT /identity/v3/projects/cf808d4541ca49eb914d7f31a78d6ef7/users/b7609bb5dc474e05b63d6566cc4c3251/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:17:41.889147 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d15b86-98a4-4b66-ac86-10eb48a437dc admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:41.900698 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d15b86-98a4-4b66-ac86-10eb48a437dc admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:41.970258 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed511ea5-0518-4cd5-b0c5-d25cbb486793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:41.971023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed511ea5-0518-4cd5-b0c5-d25cbb486793 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:41.971319 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed511ea5-0518-4cd5-b0c5-d25cbb486793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:41.971598 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed511ea5-0518-4cd5-b0c5-d25cbb486793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:41.972481 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed511ea5-0518-4cd5-b0c5-d25cbb486793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:41.974406 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed511ea5-0518-4cd5-b0c5-d25cbb486793 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:41.986253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1434/2842] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:41.991011 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fda82d-04c5-4cdc-941a-f84e51118c32 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:42.002547 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fda82d-04c5-4cdc-941a-f84e51118c32 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:42.008645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0a3b5-ec25-4ce3-8cf4-c8c2ae717685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:42.009587 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0a3b5-ec25-4ce3-8cf4-c8c2ae717685 admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:42.009858 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0a3b5-ec25-4ce3-8cf4-c8c2ae717685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:42.010138 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0a3b5-ec25-4ce3-8cf4-c8c2ae717685 admin admin] PATH_INFO: `/v3/projects/cf808d4541ca49eb914d7f31a78d6ef7/users/b7609bb5dc474e05b63d6566cc4c3251/roles/053d1730ed764973abb1490894e07681` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:42.012156 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0a3b5-ec25-4ce3-8cf4-c8c2ae717685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf808d4541ca49eb914d7f31a78d6ef7, user_id=b7609bb5dc474e05b63d6566cc4c3251, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:42.014846 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0a3b5-ec25-4ce3-8cf4-c8c2ae717685 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:42.027120 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1409/2843] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:17:41 2022] PUT /identity/v3/projects/cf808d4541ca49eb914d7f31a78d6ef7/users/b7609bb5dc474e05b63d6566cc4c3251/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:42.042243 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6da30d-0afb-42cb-80f1-27ba5f786537 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:42.042540 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6da30d-0afb-42cb-80f1-27ba5f786537 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:42.042822 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6da30d-0afb-42cb-80f1-27ba5f786537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:42.075993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-5a6da30d-0afb-42cb-80f1-27ba5f786537 None None] MFA Rules not processed for user `b7609bb5dc474e05b63d6566cc4c3251`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:42.077265 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-5a6da30d-0afb-42cb-80f1-27ba5f786537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:42.123823 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:42.161505 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1435/2844] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:42 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:42.172996 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0a3b5-ec25-4ce3-8cf4-c8c2ae717685 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:42.174741 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-9fd0a3b5-ec25-4ce3-8cf4-c8c2ae717685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:42.236117 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0a3b5-ec25-4ce3-8cf4-c8c2ae717685 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:42.240822 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea21f7-ff30-478c-9a80-17cd6b8fd41f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c53e1e801e5e40e794e4ccfff34ec988', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:42.241486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea21f7-ff30-478c-9a80-17cd6b8fd41f service cinder] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:42.241771 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea21f7-ff30-478c-9a80-17cd6b8fd41f service cinder] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:42.242049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea21f7-ff30-478c-9a80-17cd6b8fd41f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:42.251891 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea21f7-ff30-478c-9a80-17cd6b8fd41f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:42.253392 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea21f7-ff30-478c-9a80-17cd6b8fd41f service cinder] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:42.264458 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1410/2845] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:17:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:17:42.402397 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6da30d-0afb-42cb-80f1-27ba5f786537 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:42.468878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6da30d-0afb-42cb-80f1-27ba5f786537 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:42.474640 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00883da7-999d-4afd-9d8f-44de7bd76f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:42.475503 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00883da7-999d-4afd-9d8f-44de7bd76f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:42.475785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00883da7-999d-4afd-9d8f-44de7bd76f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:42.476056 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00883da7-999d-4afd-9d8f-44de7bd76f14 admin admin] PATH_INFO: `/v3/users/99f9909fd4c54b0eb47d2d07a15fc117` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:42.477267 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00883da7-999d-4afd-9d8f-44de7bd76f14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99f9909fd4c54b0eb47d2d07a15fc117)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:42.479167 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00883da7-999d-4afd-9d8f-44de7bd76f14 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:42.528623 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-00883da7-999d-4afd-9d8f-44de7bd76f14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99f9909fd4c54b0eb47d2d07a15fc117'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:42.541114 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-00883da7-999d-4afd-9d8f-44de7bd76f14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99f9909fd4c54b0eb47d2d07a15fc117'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:42.548163 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-00883da7-999d-4afd-9d8f-44de7bd76f14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99f9909fd4c54b0eb47d2d07a15fc117'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:42.557184 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-00883da7-999d-4afd-9d8f-44de7bd76f14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99f9909fd4c54b0eb47d2d07a15fc117'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:42.557981 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-00883da7-999d-4afd-9d8f-44de7bd76f14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99f9909fd4c54b0eb47d2d07a15fc117'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:42.564174 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1436/2846] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:17:42 2022] DELETE /identity/v3/users/99f9909fd4c54b0eb47d2d07a15fc117 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:42.569486 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca363b50-db09-4363-87e7-cf28ca027032 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:42.569778 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca363b50-db09-4363-87e7-cf28ca027032 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:42.570056 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca363b50-db09-4363-87e7-cf28ca027032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:42.587436 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-ca363b50-db09-4363-87e7-cf28ca027032 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:42.588651 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ca363b50-db09-4363-87e7-cf28ca027032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:42.648581 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1411/2847] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:17:42.659385 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00883da7-999d-4afd-9d8f-44de7bd76f14 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:42.662831 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-00883da7-999d-4afd-9d8f-44de7bd76f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:42.721010 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00883da7-999d-4afd-9d8f-44de7bd76f14 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:42.725370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa7999-2a14-4e11-80cd-a2f343800918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:42.726051 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa7999-2a14-4e11-80cd-a2f343800918 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:42.726332 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa7999-2a14-4e11-80cd-a2f343800918 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:42.726608 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa7999-2a14-4e11-80cd-a2f343800918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:42.733624 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:42.736057 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa7999-2a14-4e11-80cd-a2f343800918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:42.738771 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa7999-2a14-4e11-80cd-a2f343800918 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:42.750198 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1437/2848] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:17:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:17:43.028080 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca363b50-db09-4363-87e7-cf28ca027032 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:43.029615 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-ca363b50-db09-4363-87e7-cf28ca027032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:43.045694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca363b50-db09-4363-87e7-cf28ca027032 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:43.051216 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:43.052140 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:43.052434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:43.052694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2 admin admin] PATH_INFO: `/v3/projects/57ea794654fb4411b0fadacb632918a2` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:43.054481 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57ea794654fb4411b0fadacb632918a2)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:43.056798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:43.107115 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57ea794654fb4411b0fadacb632918a2', 'request_id': 'req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:43.114174 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57ea794654fb4411b0fadacb632918a2', 'request_id': 'req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:43.122710 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57ea794654fb4411b0fadacb632918a2', 'request_id': 'req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:43.134042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2 admin admin] The token cache is being invalidate because project 57ea794654fb4411b0fadacb632918a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:17:43.134334 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:43.134991 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:43.136601 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1412/2849] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:17:43 2022] DELETE /identity/v3/projects/57ea794654fb4411b0fadacb632918a2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:43.140595 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa7999-2a14-4e11-80cd-a2f343800918 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:43.150825 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa7999-2a14-4e11-80cd-a2f343800918 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:43.210048 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf83bd-68f5-4700-8ecc-5aea769a98bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:43.210937 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf83bd-68f5-4700-8ecc-5aea769a98bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:43.211252 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf83bd-68f5-4700-8ecc-5aea769a98bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:43.211557 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf83bd-68f5-4700-8ecc-5aea769a98bb admin admin] PATH_INFO: `/v3/users/b7609bb5dc474e05b63d6566cc4c3251` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:43.212847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf83bd-68f5-4700-8ecc-5aea769a98bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7609bb5dc474e05b63d6566cc4c3251)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:43.214818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf83bd-68f5-4700-8ecc-5aea769a98bb admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:43.264023 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-84bf83bd-68f5-4700-8ecc-5aea769a98bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7609bb5dc474e05b63d6566cc4c3251'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:43.275814 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-84bf83bd-68f5-4700-8ecc-5aea769a98bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7609bb5dc474e05b63d6566cc4c3251'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:43.282644 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-84bf83bd-68f5-4700-8ecc-5aea769a98bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7609bb5dc474e05b63d6566cc4c3251'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:43.291262 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-84bf83bd-68f5-4700-8ecc-5aea769a98bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7609bb5dc474e05b63d6566cc4c3251'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:43.291979 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-84bf83bd-68f5-4700-8ecc-5aea769a98bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7609bb5dc474e05b63d6566cc4c3251'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:43.298012 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1438/2850] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:17:43 2022] DELETE /identity/v3/users/b7609bb5dc474e05b63d6566cc4c3251 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:43.576855 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:43.578364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:43.587322 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:43.594860 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5eb89-fefb-44bf-bb04-bc14b4c5f3f2 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:43.658225 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9c782-13fd-4f69-97c8-feaddf12e4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:43.659089 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9c782-13fd-4f69-97c8-feaddf12e4ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:43.659370 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9c782-13fd-4f69-97c8-feaddf12e4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:43.659720 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9c782-13fd-4f69-97c8-feaddf12e4ab admin admin] PATH_INFO: `/v3/projects/cf808d4541ca49eb914d7f31a78d6ef7` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:43.661603 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9c782-13fd-4f69-97c8-feaddf12e4ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf808d4541ca49eb914d7f31a78d6ef7)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:43.663512 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9c782-13fd-4f69-97c8-feaddf12e4ab admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:43.711267 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f7f9c782-13fd-4f69-97c8-feaddf12e4ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf808d4541ca49eb914d7f31a78d6ef7', 'request_id': 'req-f7f9c782-13fd-4f69-97c8-feaddf12e4ab'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:43.717449 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f7f9c782-13fd-4f69-97c8-feaddf12e4ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf808d4541ca49eb914d7f31a78d6ef7', 'request_id': 'req-f7f9c782-13fd-4f69-97c8-feaddf12e4ab'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:43.725297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f7f9c782-13fd-4f69-97c8-feaddf12e4ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf808d4541ca49eb914d7f31a78d6ef7', 'request_id': 'req-f7f9c782-13fd-4f69-97c8-feaddf12e4ab'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:43.736146 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f7f9c782-13fd-4f69-97c8-feaddf12e4ab admin admin] The token cache is being invalidate because project cf808d4541ca49eb914d7f31a78d6ef7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:17:43.736454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f7f9c782-13fd-4f69-97c8-feaddf12e4ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:43.737075 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-f7f9c782-13fd-4f69-97c8-feaddf12e4ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:17:43.739065 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1413/2851] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:17:43 2022] DELETE /identity/v3/projects/cf808d4541ca49eb914d7f31a78d6ef7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:43.743974 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc1cb7-db49-4f1e-bffc-e56b563e8fcc None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:43.744264 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc1cb7-db49-4f1e-bffc-e56b563e8fcc None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:43.744576 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc1cb7-db49-4f1e-bffc-e56b563e8fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:43.761872 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-48cc1cb7-db49-4f1e-bffc-e56b563e8fcc None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:43.763061 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-48cc1cb7-db49-4f1e-bffc-e56b563e8fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:43.824004 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1439/2852] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:43.832672 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3acc3c-3a45-46b9-8f0f-2e554c39b0eb None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:43.832998 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3acc3c-3a45-46b9-8f0f-2e554c39b0eb None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:43.833277 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3acc3c-3a45-46b9-8f0f-2e554c39b0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:43.852124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-1f3acc3c-3a45-46b9-8f0f-2e554c39b0eb None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:43.853361 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-1f3acc3c-3a45-46b9-8f0f-2e554c39b0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:43.861271 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1414/2853] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 22:17:43.866346 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc1cb7-db49-4f1e-bffc-e56b563e8fcc None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:43.877914 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc1cb7-db49-4f1e-bffc-e56b563e8fcc None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:43.883370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58feee1b-2994-4d71-aeb3-3f49356a2474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:43.884035 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58feee1b-2994-4d71-aeb3-3f49356a2474 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:43.884321 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58feee1b-2994-4d71-aeb3-3f49356a2474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:43.884601 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58feee1b-2994-4d71-aeb3-3f49356a2474 admin admin] PATH_INFO: `/v3/domains` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:43.885373 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58feee1b-2994-4d71-aeb3-3f49356a2474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80218) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 22:17:43.885687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58feee1b-2994-4d71-aeb3-3f49356a2474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:43.887295 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58feee1b-2994-4d71-aeb3-3f49356a2474 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:43.899996 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1440/2854] 10.222.0.44 () {58 vars in 1152 bytes} [Tue Jul 12 22:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 22:17:43.905039 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3acc3c-3a45-46b9-8f0f-2e554c39b0eb None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:43.914798 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3acc3c-3a45-46b9-8f0f-2e554c39b0eb None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:43.919402 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f8d98-7fa4-4578-98e6-7419a13059ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:43.920034 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f8d98-7fa4-4578-98e6-7419a13059ec admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:43.920312 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f8d98-7fa4-4578-98e6-7419a13059ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:43.920587 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f8d98-7fa4-4578-98e6-7419a13059ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:43.921297 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f8d98-7fa4-4578-98e6-7419a13059ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:43.923187 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f8d98-7fa4-4578-98e6-7419a13059ec admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:43.949219 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1415/2855] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:17:43 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:43.953316 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58feee1b-2994-4d71-aeb3-3f49356a2474 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:43.959432 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:43.965955 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58feee1b-2994-4d71-aeb3-3f49356a2474 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:44.054689 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf85233-2f4e-4f63-9b7a-36f93c9feb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:44.055355 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf85233-2f4e-4f63-9b7a-36f93c9feb61 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:44.055639 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf85233-2f4e-4f63-9b7a-36f93c9feb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:44.055916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf85233-2f4e-4f63-9b7a-36f93c9feb61 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:44.056613 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf85233-2f4e-4f63-9b7a-36f93c9feb61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:44.058571 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf85233-2f4e-4f63-9b7a-36f93c9feb61 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:44.078878 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1441/2856] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:17:43 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:44.084061 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f8d98-7fa4-4578-98e6-7419a13059ec admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:44.094694 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f8d98-7fa4-4578-98e6-7419a13059ec admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:44.099502 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a48b5-cea4-47e1-9837-7cdfc5b21de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:44.100047 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a48b5-cea4-47e1-9837-7cdfc5b21de1 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:44.100325 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a48b5-cea4-47e1-9837-7cdfc5b21de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:44.100605 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a48b5-cea4-47e1-9837-7cdfc5b21de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:44.101365 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a48b5-cea4-47e1-9837-7cdfc5b21de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:44.102971 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a48b5-cea4-47e1-9837-7cdfc5b21de1 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:44.113897 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1416/2857] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:44.118030 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf85233-2f4e-4f63-9b7a-36f93c9feb61 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:44.127945 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf85233-2f4e-4f63-9b7a-36f93c9feb61 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:44.133119 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211b3cd-646c-4f52-8958-2100dbfb23a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:44.133747 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211b3cd-646c-4f52-8958-2100dbfb23a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:44.134031 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211b3cd-646c-4f52-8958-2100dbfb23a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:44.134309 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211b3cd-646c-4f52-8958-2100dbfb23a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:44.135061 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211b3cd-646c-4f52-8958-2100dbfb23a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:44.136651 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211b3cd-646c-4f52-8958-2100dbfb23a3 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:44.147471 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1442/2858] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:44.151563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a48b5-cea4-47e1-9837-7cdfc5b21de1 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:44.161862 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a48b5-cea4-47e1-9837-7cdfc5b21de1 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:44.166494 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150e490-ef73-4539-a45f-929e4f331e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:44.167364 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150e490-ef73-4539-a45f-929e4f331e4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:44.167649 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150e490-ef73-4539-a45f-929e4f331e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:44.167926 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150e490-ef73-4539-a45f-929e4f331e4f admin admin] PATH_INFO: `/v3/projects/43ac35d1693a46e49410be387243925e/users/fe3daae99f8143aca14bced56cb6c131/roles/b5ee21738ecb4079977d078d50ddbcf9` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:44.182463 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150e490-ef73-4539-a45f-929e4f331e4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=43ac35d1693a46e49410be387243925e, user_id=fe3daae99f8143aca14bced56cb6c131, role_id=b5ee21738ecb4079977d078d50ddbcf9)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:44.185201 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150e490-ef73-4539-a45f-929e4f331e4f admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:44.199332 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1417/2859] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:17:44 2022] PUT /identity/v3/projects/43ac35d1693a46e49410be387243925e/users/fe3daae99f8143aca14bced56cb6c131/roles/b5ee21738ecb4079977d078d50ddbcf9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:44.204378 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fbd8e-1ac9-4c5f-8aba-221e04aa65e3 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:44.204687 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fbd8e-1ac9-4c5f-8aba-221e04aa65e3 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:44.204999 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fbd8e-1ac9-4c5f-8aba-221e04aa65e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:44.223972 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-cc5fbd8e-1ac9-4c5f-8aba-221e04aa65e3 None None] MFA Rules not processed for user `a08b779b7e9045028b1b3a86939d306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:44.225138 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-cc5fbd8e-1ac9-4c5f-8aba-221e04aa65e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:44.285061 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1443/2860] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:44.294479 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150e490-ef73-4539-a45f-929e4f331e4f admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:44.296177 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6150e490-ef73-4539-a45f-929e4f331e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:44.329817 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:44.356563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150e490-ef73-4539-a45f-929e4f331e4f admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:44.360199 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339ee7b-7eef-493e-a6c2-5e6fa4c0cd3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:44.360871 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339ee7b-7eef-493e-a6c2-5e6fa4c0cd3a service neutron] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:44.361151 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339ee7b-7eef-493e-a6c2-5e6fa4c0cd3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:44.361425 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339ee7b-7eef-493e-a6c2-5e6fa4c0cd3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:44.370964 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339ee7b-7eef-493e-a6c2-5e6fa4c0cd3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:44.372388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339ee7b-7eef-493e-a6c2-5e6fa4c0cd3a service neutron] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:44.382325 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1418/2861] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:17:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:17:49.830173 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a12cd-e28c-4b99-962a-c427a12cd6c5 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:49.830718 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a12cd-e28c-4b99-962a-c427a12cd6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:49.830893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a12cd-e28c-4b99-962a-c427a12cd6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:49.863235 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-a44a12cd-e28c-4b99-962a-c427a12cd6c5 None None] MFA Rules not processed for user `fe3daae99f8143aca14bced56cb6c131`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:49.864859 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-a44a12cd-e28c-4b99-962a-c427a12cd6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:49.951492 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1444/2862] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:49.952886 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339ee7b-7eef-493e-a6c2-5e6fa4c0cd3a service neutron] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:49.971124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339ee7b-7eef-493e-a6c2-5e6fa4c0cd3a service neutron] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:49.979021 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffc4ae-2365-47e6-8d06-d5dacd768d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:49.980700 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffc4ae-2365-47e6-8d06-d5dacd768d38 admin admin] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:49.981068 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffc4ae-2365-47e6-8d06-d5dacd768d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:49.981454 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffc4ae-2365-47e6-8d06-d5dacd768d38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:49.982049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffc4ae-2365-47e6-8d06-d5dacd768d38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:49.983969 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffc4ae-2365-47e6-8d06-d5dacd768d38 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:50.019042 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1419/2863] 10.222.0.44 () {60 vars in 1152 bytes} [Tue Jul 12 22:17:49 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:50.023852 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a12cd-e28c-4b99-962a-c427a12cd6c5 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:50.045732 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a12cd-e28c-4b99-962a-c427a12cd6c5 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:50.058203 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:50.133411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055d839-2042-4a09-baa7-5b28813b99e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:50.134253 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055d839-2042-4a09-baa7-5b28813b99e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:50.135497 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055d839-2042-4a09-baa7-5b28813b99e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:50.135789 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055d839-2042-4a09-baa7-5b28813b99e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:50.136655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055d839-2042-4a09-baa7-5b28813b99e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:50.138683 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055d839-2042-4a09-baa7-5b28813b99e1 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:50.163626 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1445/2864] 10.222.0.44 () {60 vars in 1143 bytes} [Tue Jul 12 22:17:50 2022] POST /identity/v3/users => generated 298 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 22:17:50.168045 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffc4ae-2365-47e6-8d06-d5dacd768d38 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:50.181839 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffc4ae-2365-47e6-8d06-d5dacd768d38 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:50.189124 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe91184-2fae-4bcc-aacb-e8c424ad25b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:50.190022 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe91184-2fae-4bcc-aacb-e8c424ad25b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:50.190390 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe91184-2fae-4bcc-aacb-e8c424ad25b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:50.191354 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe91184-2fae-4bcc-aacb-e8c424ad25b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:50.192352 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe91184-2fae-4bcc-aacb-e8c424ad25b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:50.194585 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe91184-2fae-4bcc-aacb-e8c424ad25b7 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:50.212342 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1420/2865] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:50.216595 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055d839-2042-4a09-baa7-5b28813b99e1 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:50.228397 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055d839-2042-4a09-baa7-5b28813b99e1 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:50.235122 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2267da1-ce74-485c-9187-01fc4e79a85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:50.236139 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2267da1-ce74-485c-9187-01fc4e79a85b admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:50.236415 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2267da1-ce74-485c-9187-01fc4e79a85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:50.236697 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2267da1-ce74-485c-9187-01fc4e79a85b admin admin] PATH_INFO: `/v3/projects/a342b9e8a56142c3b1d2dc43c3420da9/users/dd7c7d9bd73b41dfb084011e306ab954/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:50.254129 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2267da1-ce74-485c-9187-01fc4e79a85b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a342b9e8a56142c3b1d2dc43c3420da9, user_id=dd7c7d9bd73b41dfb084011e306ab954, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:50.256835 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2267da1-ce74-485c-9187-01fc4e79a85b admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:50.273509 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1446/2866] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:17:50 2022] PUT /identity/v3/projects/a342b9e8a56142c3b1d2dc43c3420da9/users/dd7c7d9bd73b41dfb084011e306ab954/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:50.277686 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe91184-2fae-4bcc-aacb-e8c424ad25b7 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:50.288262 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe91184-2fae-4bcc-aacb-e8c424ad25b7 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:50.348978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac3586-862e-4506-877a-e0c7c8fe0034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:50.349656 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac3586-862e-4506-877a-e0c7c8fe0034 admin admin] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:50.350365 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac3586-862e-4506-877a-e0c7c8fe0034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:50.350365 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac3586-862e-4506-877a-e0c7c8fe0034 admin admin] PATH_INFO: `/v3/roles` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:50.350929 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac3586-862e-4506-877a-e0c7c8fe0034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:50.352529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac3586-862e-4506-877a-e0c7c8fe0034 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:50.364464 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1421/2867] 10.222.0.44 () {58 vars in 1121 bytes} [Tue Jul 12 22:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 22:17:50.368411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2267da1-ce74-485c-9187-01fc4e79a85b admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:50.385537 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2267da1-ce74-485c-9187-01fc4e79a85b admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:50.391107 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132658b-1494-4412-911c-8c093eda4ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:50.392348 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132658b-1494-4412-911c-8c093eda4ad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:50.392678 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132658b-1494-4412-911c-8c093eda4ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:50.393084 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132658b-1494-4412-911c-8c093eda4ad7 admin admin] PATH_INFO: `/v3/projects/a342b9e8a56142c3b1d2dc43c3420da9/users/dd7c7d9bd73b41dfb084011e306ab954/roles/053d1730ed764973abb1490894e07681` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:50.395681 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132658b-1494-4412-911c-8c093eda4ad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a342b9e8a56142c3b1d2dc43c3420da9, user_id=dd7c7d9bd73b41dfb084011e306ab954, role_id=053d1730ed764973abb1490894e07681)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:50.400608 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132658b-1494-4412-911c-8c093eda4ad7 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:50.414103 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1447/2868] 10.222.0.44 () {60 vars in 1482 bytes} [Tue Jul 12 22:17:50 2022] PUT /identity/v3/projects/a342b9e8a56142c3b1d2dc43c3420da9/users/dd7c7d9bd73b41dfb084011e306ab954/roles/053d1730ed764973abb1490894e07681 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:17:56.729515 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d348e17-15df-465a-800f-5015c8a894ce None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:56.730518 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d348e17-15df-465a-800f-5015c8a894ce None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:56.731017 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d348e17-15df-465a-800f-5015c8a894ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:56.768206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-8d348e17-15df-465a-800f-5015c8a894ce None None] MFA Rules not processed for user `dd7c7d9bd73b41dfb084011e306ab954`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:56.770476 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8d348e17-15df-465a-800f-5015c8a894ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:56.852625 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132658b-1494-4412-911c-8c093eda4ad7 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:56.854485 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-0132658b-1494-4412-911c-8c093eda4ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:56.899211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1422/2869] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:17:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:56.910841 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d348e17-15df-465a-800f-5015c8a894ce None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:56.912946 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8d348e17-15df-465a-800f-5015c8a894ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:56.973563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:57.013340 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132658b-1494-4412-911c-8c093eda4ad7 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:57.028677 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad3621-8bab-4baa-b07e-7fc2b7bb88aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f1ef52f8eb42a59a50a69873b9ffaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:57.029462 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad3621-8bab-4baa-b07e-7fc2b7bb88aa service placement] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:57.029962 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad3621-8bab-4baa-b07e-7fc2b7bb88aa service placement] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:57.030243 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad3621-8bab-4baa-b07e-7fc2b7bb88aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:57.032220 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-00ad3621-8bab-4baa-b07e-7fc2b7bb88aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:57.062191 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad3621-8bab-4baa-b07e-7fc2b7bb88aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:57.063801 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad3621-8bab-4baa-b07e-7fc2b7bb88aa service placement] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:57.097460 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1448/2870] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:17:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:17:57.133529 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d348e17-15df-465a-800f-5015c8a894ce None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:57.137730 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ccf86-6ce6-44e8-a8c4-51ca9ba1bf1c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5c9ed6f9fd542069cf914003b636c1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:57.138549 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ccf86-6ce6-44e8-a8c4-51ca9ba1bf1c service glance] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:57.138957 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ccf86-6ce6-44e8-a8c4-51ca9ba1bf1c service glance] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:57.139336 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ccf86-6ce6-44e8-a8c4-51ca9ba1bf1c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:57.250864 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ccf86-6ce6-44e8-a8c4-51ca9ba1bf1c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:57.252491 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ccf86-6ce6-44e8-a8c4-51ca9ba1bf1c service glance] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:57.266217 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1423/2871] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:17:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:17:57.416758 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a0f0c-bc45-4138-95ae-002a331dd529 None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:57.417311 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a0f0c-bc45-4138-95ae-002a331dd529 None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:57.417599 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a0f0c-bc45-4138-95ae-002a331dd529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:57.493845 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-d74a0f0c-bc45-4138-95ae-002a331dd529 None None] MFA Rules not processed for user `8d6b823cc7dc434986f77739fa0259ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:17:57.495384 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d74a0f0c-bc45-4138-95ae-002a331dd529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:57.512562 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:17:57.602430 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1449/2872] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Jul 12 22:17:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:17:57.656386 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ccf86-6ce6-44e8-a8c4-51ca9ba1bf1c service glance] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:17:57.658039 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-da3ccf86-6ce6-44e8-a8c4-51ca9ba1bf1c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:17:57.786810 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ccf86-6ce6-44e8-a8c4-51ca9ba1bf1c service glance] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:17:57.793237 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341a26b-2e1d-4ad7-9d8c-8fa98a8df7fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837bb5ac18554dfdbbcc776689639494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:17:57.793990 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341a26b-2e1d-4ad7-9d8c-8fa98a8df7fb service swift] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:17:57.794488 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341a26b-2e1d-4ad7-9d8c-8fa98a8df7fb service swift] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:17:57.794978 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341a26b-2e1d-4ad7-9d8c-8fa98a8df7fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:17:57.808204 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341a26b-2e1d-4ad7-9d8c-8fa98a8df7fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:17:57.809990 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341a26b-2e1d-4ad7-9d8c-8fa98a8df7fb service swift] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:17:57.822343 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1424/2873] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Jul 12 22:17:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 22:18:00.902005 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a0f0c-bc45-4138-95ae-002a331dd529 None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:18:00.903906 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-d74a0f0c-bc45-4138-95ae-002a331dd529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:18:01.031483 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a0f0c-bc45-4138-95ae-002a331dd529 None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:18:01.045786 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0da0d6-a828-49c0-bcde-704b447d272b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:18:01.046691 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0da0d6-a828-49c0-bcde-704b447d272b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:18:01.046973 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0da0d6-a828-49c0-bcde-704b447d272b admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:18:01.047254 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0da0d6-a828-49c0-bcde-704b447d272b admin admin] PATH_INFO: `/v3/users/624bdc53a8b34b46b5004532716b3985` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:18:01.055070 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0da0d6-a828-49c0-bcde-704b447d272b admin admin] RBAC: Authorizing `identity:delete_user(user_id=624bdc53a8b34b46b5004532716b3985)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:18:01.057065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0da0d6-a828-49c0-bcde-704b447d272b admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:18:01.137041 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ea0da0d6-a828-49c0-bcde-704b447d272b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '624bdc53a8b34b46b5004532716b3985'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:01.157669 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ea0da0d6-a828-49c0-bcde-704b447d272b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '624bdc53a8b34b46b5004532716b3985'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:01.179549 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ea0da0d6-a828-49c0-bcde-704b447d272b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '624bdc53a8b34b46b5004532716b3985'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:01.198648 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ea0da0d6-a828-49c0-bcde-704b447d272b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '624bdc53a8b34b46b5004532716b3985'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:01.200343 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-ea0da0d6-a828-49c0-bcde-704b447d272b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '624bdc53a8b34b46b5004532716b3985'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:01.211443 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1450/2874] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:18:00 2022] DELETE /identity/v3/users/624bdc53a8b34b46b5004532716b3985 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:18:01.525150 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341a26b-2e1d-4ad7-9d8c-8fa98a8df7fb service swift] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:18:01.539801 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341a26b-2e1d-4ad7-9d8c-8fa98a8df7fb service swift] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:18:01.571438 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=100 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:18:01.624629 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd2899-5872-4799-8baf-c354acc0b956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:18:01.625709 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd2899-5872-4799-8baf-c354acc0b956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:18:01.625989 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd2899-5872-4799-8baf-c354acc0b956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:18:01.626740 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd2899-5872-4799-8baf-c354acc0b956 admin admin] PATH_INFO: `/v3/projects/b46992df6b784a7abd397b8f3aeb827a` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:18:01.630650 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd2899-5872-4799-8baf-c354acc0b956 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b46992df6b784a7abd397b8f3aeb827a)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:18:01.632681 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd2899-5872-4799-8baf-c354acc0b956 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:18:01.712677 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-47bd2899-5872-4799-8baf-c354acc0b956 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b46992df6b784a7abd397b8f3aeb827a', 'request_id': 'req-47bd2899-5872-4799-8baf-c354acc0b956'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:01.719873 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-47bd2899-5872-4799-8baf-c354acc0b956 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b46992df6b784a7abd397b8f3aeb827a', 'request_id': 'req-47bd2899-5872-4799-8baf-c354acc0b956'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:01.740680 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-47bd2899-5872-4799-8baf-c354acc0b956 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b46992df6b784a7abd397b8f3aeb827a', 'request_id': 'req-47bd2899-5872-4799-8baf-c354acc0b956'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:01.757910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-47bd2899-5872-4799-8baf-c354acc0b956 admin admin] The token cache is being invalidate because project b46992df6b784a7abd397b8f3aeb827a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:18:01.757910 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-47bd2899-5872-4799-8baf-c354acc0b956 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:01.758522 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-47bd2899-5872-4799-8baf-c354acc0b956 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:01.760341 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1425/2875] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:18:01 2022] DELETE /identity/v3/projects/b46992df6b784a7abd397b8f3aeb827a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:18:01.764650 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0da0d6-a828-49c0-bcde-704b447d272b admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:18:01.766360 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-ea0da0d6-a828-49c0-bcde-704b447d272b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:18:01.790900 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0da0d6-a828-49c0-bcde-704b447d272b admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:18:01.852892 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:18:01.921686 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4152a-e6a2-47ad-94f3-c0c5529dea6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:18:01.924161 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4152a-e6a2-47ad-94f3-c0c5529dea6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:18:01.924470 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4152a-e6a2-47ad-94f3-c0c5529dea6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:18:01.925022 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4152a-e6a2-47ad-94f3-c0c5529dea6a admin admin] PATH_INFO: `/v3/users/8a14af7da1bd49e7bc3dd258a42ad010` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:18:01.926502 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4152a-e6a2-47ad-94f3-c0c5529dea6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a14af7da1bd49e7bc3dd258a42ad010)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:18:01.937039 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4152a-e6a2-47ad-94f3-c0c5529dea6a admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:18:02.030203 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-39f4152a-e6a2-47ad-94f3-c0c5529dea6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a14af7da1bd49e7bc3dd258a42ad010'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:02.051705 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-39f4152a-e6a2-47ad-94f3-c0c5529dea6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a14af7da1bd49e7bc3dd258a42ad010'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:02.068107 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-39f4152a-e6a2-47ad-94f3-c0c5529dea6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a14af7da1bd49e7bc3dd258a42ad010'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:02.090325 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-39f4152a-e6a2-47ad-94f3-c0c5529dea6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a14af7da1bd49e7bc3dd258a42ad010'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:02.091058 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-39f4152a-e6a2-47ad-94f3-c0c5529dea6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a14af7da1bd49e7bc3dd258a42ad010'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:02.110138 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1451/2876] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:18:01 2022] DELETE /identity/v3/users/8a14af7da1bd49e7bc3dd258a42ad010 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:18:02.349305 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd2899-5872-4799-8baf-c354acc0b956 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:18:02.350850 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-47bd2899-5872-4799-8baf-c354acc0b956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:18:02.369668 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd2899-5872-4799-8baf-c354acc0b956 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:18:02.469869 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d38abd-b3b5-4333-95cf-ec96a8e64515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:18:02.470928 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d38abd-b3b5-4333-95cf-ec96a8e64515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:18:02.471211 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d38abd-b3b5-4333-95cf-ec96a8e64515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:18:02.471483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d38abd-b3b5-4333-95cf-ec96a8e64515 admin admin] PATH_INFO: `/v3/projects/f5ecc2b62def4a579ad605653b167096` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:18:02.472968 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d38abd-b3b5-4333-95cf-ec96a8e64515 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5ecc2b62def4a579ad605653b167096)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:18:02.474886 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d38abd-b3b5-4333-95cf-ec96a8e64515 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:18:02.583596 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-53d38abd-b3b5-4333-95cf-ec96a8e64515 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5ecc2b62def4a579ad605653b167096', 'request_id': 'req-53d38abd-b3b5-4333-95cf-ec96a8e64515'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:02.594795 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-53d38abd-b3b5-4333-95cf-ec96a8e64515 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5ecc2b62def4a579ad605653b167096', 'request_id': 'req-53d38abd-b3b5-4333-95cf-ec96a8e64515'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:02.599186 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:18:02.610775 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-53d38abd-b3b5-4333-95cf-ec96a8e64515 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5ecc2b62def4a579ad605653b167096', 'request_id': 'req-53d38abd-b3b5-4333-95cf-ec96a8e64515'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:02.628679 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-53d38abd-b3b5-4333-95cf-ec96a8e64515 admin admin] The token cache is being invalidate because project f5ecc2b62def4a579ad605653b167096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80219) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:18:02.629091 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-53d38abd-b3b5-4333-95cf-ec96a8e64515 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:02.629914 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-53d38abd-b3b5-4333-95cf-ec96a8e64515 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:18:02.634218 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1426/2877] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:18:02 2022] DELETE /identity/v3/projects/f5ecc2b62def4a579ad605653b167096 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:18:07.584172 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade54e1-77af-4ffd-ae6d-b4449fce49ad None None] REQUEST_METHOD: `POST` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:18:07.584695 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade54e1-77af-4ffd-ae6d-b4449fce49ad None None] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:18:07.584818 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade54e1-77af-4ffd-ae6d-b4449fce49ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:18:07.602608 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.auth.core [None req-aade54e1-77af-4ffd-ae6d-b4449fce49ad None None] MFA Rules not processed for user `fe3daae99f8143aca14bced56cb6c131`. Rule list: `[]` (Enabled: `True`). {{(pid=80218) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:18:07.603819 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-aade54e1-77af-4ffd-ae6d-b4449fce49ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:18:07.661376 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1452/2878] 10.222.0.44 () {58 vars in 957 bytes} [Tue Jul 12 22:18:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:18:09.372317 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e468f78-d754-4d9d-a17d-abad0347b441 None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:18:09.372889 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e468f78-d754-4d9d-a17d-abad0347b441 None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:18:09.372983 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e468f78-d754-4d9d-a17d-abad0347b441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:18:09.390814 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-8e468f78-d754-4d9d-a17d-abad0347b441 None None] MFA Rules not processed for user `8d6b823cc7dc434986f77739fa0259ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:18:09.392049 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8e468f78-d754-4d9d-a17d-abad0347b441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:18:09.449132 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1427/2879] 10.222.0.44 () {58 vars in 1009 bytes} [Tue Jul 12 22:18:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:18:09.464429 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade54e1-77af-4ffd-ae6d-b4449fce49ad None None] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:18:09.466129 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-aade54e1-77af-4ffd-ae6d-b4449fce49ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:18:09.661567 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:18:09.668960 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade54e1-77af-4ffd-ae6d-b4449fce49ad None None] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:18:09.675025 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76468943-134f-48a6-963e-22de7f505ec1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837bb5ac18554dfdbbcc776689639494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:18:09.675709 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76468943-134f-48a6-963e-22de7f505ec1 service swift] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:18:09.676101 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76468943-134f-48a6-963e-22de7f505ec1 service swift] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:18:09.676610 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76468943-134f-48a6-963e-22de7f505ec1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:18:09.685483 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76468943-134f-48a6-963e-22de7f505ec1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:18:09.687173 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76468943-134f-48a6-963e-22de7f505ec1 service swift] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:18:09.696840 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1453/2880] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Jul 12 22:18:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 22:18:19.441856 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:DELETE=3,keystone:SELECT=29 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:18:19.693589 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=5 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:18:22.155570 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e468f78-d754-4d9d-a17d-abad0347b441 None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:18:22.157157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-8e468f78-d754-4d9d-a17d-abad0347b441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:18:22.235715 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e468f78-d754-4d9d-a17d-abad0347b441 None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:18:22.241279 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a63365-19d7-4b2b-8853-9accb0e04b2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:18:22.241921 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a63365-19d7-4b2b-8853-9accb0e04b2b service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:18:22.242206 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a63365-19d7-4b2b-8853-9accb0e04b2b service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:18:22.242483 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a63365-19d7-4b2b-8853-9accb0e04b2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:18:22.244302 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-06a63365-19d7-4b2b-8853-9accb0e04b2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:18:22.257666 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a63365-19d7-4b2b-8853-9accb0e04b2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:18:22.259272 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a63365-19d7-4b2b-8853-9accb0e04b2b service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:18:22.270172 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1428/2881] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:18:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:18:32.265584 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=34 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:18:33.809683 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76468943-134f-48a6-963e-22de7f505ec1 service swift] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:18:33.811125 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-76468943-134f-48a6-963e-22de7f505ec1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:18:33.869249 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76468943-134f-48a6-963e-22de7f505ec1 service swift] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:18:33.873061 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2cbf7-ae17-4b59-8a67-df08972baa39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53b891a734a64b5983564e207de0359d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:18:33.873709 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2cbf7-ae17-4b59-8a67-df08972baa39 service neutron] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:18:33.873998 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2cbf7-ae17-4b59-8a67-df08972baa39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:18:33.874276 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2cbf7-ae17-4b59-8a67-df08972baa39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:18:33.876095 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-38a2cbf7-ae17-4b59-8a67-df08972baa39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:18:33.938848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2cbf7-ae17-4b59-8a67-df08972baa39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:18:33.940280 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2cbf7-ae17-4b59-8a67-df08972baa39 service neutron] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:18:33.950854 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1454/2882] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:18:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:18:35.805647 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a63365-19d7-4b2b-8853-9accb0e04b2b service nova] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:18:35.806864 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-06a63365-19d7-4b2b-8853-9accb0e04b2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:18:35.822433 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a63365-19d7-4b2b-8853-9accb0e04b2b service nova] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:18:35.828959 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ad201-a38f-432c-9392-7dfdfbd05fa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24915625354f418eb7bc4d5ada85fdb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:18:35.829657 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ad201-a38f-432c-9392-7dfdfbd05fa6 service nova] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:18:35.829942 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ad201-a38f-432c-9392-7dfdfbd05fa6 service nova] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:18:35.830221 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ad201-a38f-432c-9392-7dfdfbd05fa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:18:35.832038 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-2d6ad201-a38f-432c-9392-7dfdfbd05fa6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:18:35.846387 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ad201-a38f-432c-9392-7dfdfbd05fa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:18:35.847992 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ad201-a38f-432c-9392-7dfdfbd05fa6 service nova] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:18:35.860786 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1429/2883] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:18:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:18:43.946655 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=50 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:18:45.855536 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=6 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:18:54.530731 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2cbf7-ae17-4b59-8a67-df08972baa39 service neutron] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:18:54.532160 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-38a2cbf7-ae17-4b59-8a67-df08972baa39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:18:54.602702 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2cbf7-ae17-4b59-8a67-df08972baa39 service neutron] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:18:54.607630 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732541b-f602-4c03-acf0-9138bdbfeb64 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f1ef52f8eb42a59a50a69873b9ffaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:18:54.608276 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732541b-f602-4c03-acf0-9138bdbfeb64 service placement] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:18:54.608562 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732541b-f602-4c03-acf0-9138bdbfeb64 service placement] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:18:54.608877 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732541b-f602-4c03-acf0-9138bdbfeb64 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:18:54.610631 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-2732541b-f602-4c03-acf0-9138bdbfeb64 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:18:54.623065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732541b-f602-4c03-acf0-9138bdbfeb64 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:18:54.624630 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732541b-f602-4c03-acf0-9138bdbfeb64 service placement] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:18:54.635493 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1455/2884] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:18:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:19:04.631128 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=32 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:19:47.363182 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515ed81-c8f5-4830-a307-9b78033bb6ee None None] REQUEST_METHOD: `POST` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:19:47.363761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515ed81-c8f5-4830-a307-9b78033bb6ee None None] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:19:47.363761 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515ed81-c8f5-4830-a307-9b78033bb6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:19:47.381947 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.auth.core [None req-c515ed81-c8f5-4830-a307-9b78033bb6ee None None] MFA Rules not processed for user `8d6b823cc7dc434986f77739fa0259ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80219) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 22:19:47.383973 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c515ed81-c8f5-4830-a307-9b78033bb6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:19:47.391575 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1430/2885] 10.222.0.44 () {58 vars in 967 bytes} [Tue Jul 12 22:19:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 22:19:47.432526 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732541b-f602-4c03-acf0-9138bdbfeb64 service placement] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:19:47.444133 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732541b-f602-4c03-acf0-9138bdbfeb64 service placement] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:19:47.449904 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fc03d-ed76-4612-82f9-3dafb0f20a42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '837bb5ac18554dfdbbcc776689639494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:19:47.450565 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fc03d-ed76-4612-82f9-3dafb0f20a42 service swift] REQUEST_METHOD: `GET` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:19:47.450847 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fc03d-ed76-4612-82f9-3dafb0f20a42 service swift] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:19:47.451125 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fc03d-ed76-4612-82f9-3dafb0f20a42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:19:47.459961 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fc03d-ed76-4612-82f9-3dafb0f20a42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:19:47.462200 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fc03d-ed76-4612-82f9-3dafb0f20a42 service swift] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:19:47.470658 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1456/2886] 10.222.0.44 () {60 vars in 1384 bytes} [Tue Jul 12 22:19:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 22:19:54.252732 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515ed81-c8f5-4830-a307-9b78033bb6ee None None] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:19:54.254301 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-c515ed81-c8f5-4830-a307-9b78033bb6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:19:54.329233 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515ed81-c8f5-4830-a307-9b78033bb6ee None None] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:19:54.334090 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711e9a3-9f9e-478f-9bc9-5c125d27325f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:19:54.334970 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711e9a3-9f9e-478f-9bc9-5c125d27325f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:19:54.335867 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711e9a3-9f9e-478f-9bc9-5c125d27325f admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:19:54.336157 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711e9a3-9f9e-478f-9bc9-5c125d27325f admin admin] PATH_INFO: `/v3/users/fe3daae99f8143aca14bced56cb6c131` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:19:54.337410 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711e9a3-9f9e-478f-9bc9-5c125d27325f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe3daae99f8143aca14bced56cb6c131)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:19:54.339350 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711e9a3-9f9e-478f-9bc9-5c125d27325f admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:19:54.388943 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6711e9a3-9f9e-478f-9bc9-5c125d27325f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe3daae99f8143aca14bced56cb6c131'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:54.395745 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6711e9a3-9f9e-478f-9bc9-5c125d27325f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe3daae99f8143aca14bced56cb6c131'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:54.403645 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6711e9a3-9f9e-478f-9bc9-5c125d27325f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe3daae99f8143aca14bced56cb6c131'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:54.404348 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6711e9a3-9f9e-478f-9bc9-5c125d27325f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe3daae99f8143aca14bced56cb6c131'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:54.405099 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-6711e9a3-9f9e-478f-9bc9-5c125d27325f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe3daae99f8143aca14bced56cb6c131'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:54.424388 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1431/2887] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:19:54 2022] DELETE /identity/v3/users/fe3daae99f8143aca14bced56cb6c131 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:19:54.603379 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fc03d-ed76-4612-82f9-3dafb0f20a42 service swift] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:19:54.616895 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fc03d-ed76-4612-82f9-3dafb0f20a42 service swift] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:19:54.678086 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b828bd-b5f2-4d71-b1c6-944245078b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:19:54.679059 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b828bd-b5f2-4d71-b1c6-944245078b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:19:54.679411 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b828bd-b5f2-4d71-b1c6-944245078b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:19:54.679558 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b828bd-b5f2-4d71-b1c6-944245078b44 admin admin] PATH_INFO: `/v3/projects/43ac35d1693a46e49410be387243925e` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:19:54.680641 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b828bd-b5f2-4d71-b1c6-944245078b44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43ac35d1693a46e49410be387243925e)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:19:54.682559 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b828bd-b5f2-4d71-b1c6-944245078b44 admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:19:54.728438 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-b3b828bd-b5f2-4d71-b1c6-944245078b44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43ac35d1693a46e49410be387243925e', 'request_id': 'req-b3b828bd-b5f2-4d71-b1c6-944245078b44'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:54.733914 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-b3b828bd-b5f2-4d71-b1c6-944245078b44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43ac35d1693a46e49410be387243925e', 'request_id': 'req-b3b828bd-b5f2-4d71-b1c6-944245078b44'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:54.741221 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-b3b828bd-b5f2-4d71-b1c6-944245078b44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43ac35d1693a46e49410be387243925e', 'request_id': 'req-b3b828bd-b5f2-4d71-b1c6-944245078b44'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:54.752484 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-b3b828bd-b5f2-4d71-b1c6-944245078b44 admin admin] The token cache is being invalidate because project 43ac35d1693a46e49410be387243925e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:19:54.752810 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-b3b828bd-b5f2-4d71-b1c6-944245078b44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:54.753456 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-b3b828bd-b5f2-4d71-b1c6-944245078b44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:54.754993 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1457/2888] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:19:54 2022] DELETE /identity/v3/projects/43ac35d1693a46e49410be387243925e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 22:19:54.759197 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711e9a3-9f9e-478f-9bc9-5c125d27325f admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:19:54.760634 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-6711e9a3-9f9e-478f-9bc9-5c125d27325f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:19:54.774210 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711e9a3-9f9e-478f-9bc9-5c125d27325f admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:19:54.832687 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65217930-80bb-407d-8793-507ea8f45a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:19:54.833587 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65217930-80bb-407d-8793-507ea8f45a20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:19:54.833874 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65217930-80bb-407d-8793-507ea8f45a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:19:54.834156 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65217930-80bb-407d-8793-507ea8f45a20 admin admin] PATH_INFO: `/v3/users/dd7c7d9bd73b41dfb084011e306ab954` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:19:54.835434 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65217930-80bb-407d-8793-507ea8f45a20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd7c7d9bd73b41dfb084011e306ab954)` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:19:54.837480 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65217930-80bb-407d-8793-507ea8f45a20 admin admin] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:19:54.852892 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:19:54.888918 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-65217930-80bb-407d-8793-507ea8f45a20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd7c7d9bd73b41dfb084011e306ab954'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:54.895806 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-65217930-80bb-407d-8793-507ea8f45a20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd7c7d9bd73b41dfb084011e306ab954'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:54.903644 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-65217930-80bb-407d-8793-507ea8f45a20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd7c7d9bd73b41dfb084011e306ab954'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:54.904563 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-65217930-80bb-407d-8793-507ea8f45a20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd7c7d9bd73b41dfb084011e306ab954'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:54.905509 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.notifications [None req-65217930-80bb-407d-8793-507ea8f45a20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd7c7d9bd73b41dfb084011e306ab954'} {{(pid=80219) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:54.924643 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1432/2889] 10.222.0.44 () {58 vars in 1223 bytes} [Tue Jul 12 22:19:54 2022] DELETE /identity/v3/users/dd7c7d9bd73b41dfb084011e306ab954 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:19:55.103065 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b828bd-b5f2-4d71-b1c6-944245078b44 admin admin] Authenticating user token {{(pid=80218) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:19:55.104945 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.fernet_utils [None req-b3b828bd-b5f2-4d71-b1c6-944245078b44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80218) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:19:55.118893 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b828bd-b5f2-4d71-b1c6-944245078b44 admin admin] Validating token access rules against request {{(pid=80218) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:19:55.175848 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61de57d-1065-46b4-88a8-486838a34faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08b779b7e9045028b1b3a86939d306f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e825823085549c0909b324f2e7acb2a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80218) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:19:55.176736 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61de57d-1065-46b4-88a8-486838a34faf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:19:55.177032 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61de57d-1065-46b4-88a8-486838a34faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:19:55.177314 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61de57d-1065-46b4-88a8-486838a34faf admin admin] PATH_INFO: `/v3/projects/a342b9e8a56142c3b1d2dc43c3420da9` {{(pid=80218) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:19:55.178501 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61de57d-1065-46b4-88a8-486838a34faf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a342b9e8a56142c3b1d2dc43c3420da9)` {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:19:55.180750 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61de57d-1065-46b4-88a8-486838a34faf admin admin] RBAC: Authorization granted {{(pid=80218) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:19:55.201642 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:19:55.228785 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f61de57d-1065-46b4-88a8-486838a34faf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a342b9e8a56142c3b1d2dc43c3420da9', 'request_id': 'req-f61de57d-1065-46b4-88a8-486838a34faf'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:55.234213 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f61de57d-1065-46b4-88a8-486838a34faf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a342b9e8a56142c3b1d2dc43c3420da9', 'request_id': 'req-f61de57d-1065-46b4-88a8-486838a34faf'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:55.242127 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f61de57d-1065-46b4-88a8-486838a34faf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a342b9e8a56142c3b1d2dc43c3420da9', 'request_id': 'req-f61de57d-1065-46b4-88a8-486838a34faf'} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:55.254370 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f61de57d-1065-46b4-88a8-486838a34faf admin admin] The token cache is being invalidate because project a342b9e8a56142c3b1d2dc43c3420da9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80218) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 22:19:55.254683 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f61de57d-1065-46b4-88a8-486838a34faf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:55.255347 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG keystone.notifications [None req-f61de57d-1065-46b4-88a8-486838a34faf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80218) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 22:19:55.256916 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: [pid: 80218|app: 0|req: 1458/2890] 10.222.0.44 () {58 vars in 1232 bytes} [Tue Jul 12 22:19:55 2022] DELETE /identity/v3/projects/a342b9e8a56142c3b1d2dc43c3420da9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 22:20:04.921108 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=21,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:20:05.251130 dv-cinder-802955-7-iscsi devstack@keystone.service[80218]: DEBUG dbcounter [-] [80218] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80218) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 22:24:00.487583 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65217930-80bb-407d-8793-507ea8f45a20 admin admin] Authenticating user token {{(pid=80219) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 22:24:00.489125 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-65217930-80bb-407d-8793-507ea8f45a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:24:00.619130 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65217930-80bb-407d-8793-507ea8f45a20 admin admin] Validating token access rules against request {{(pid=80219) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 22:24:00.678575 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6213d-46bb-46e3-a1e8-01320bf09164 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f1ef52f8eb42a59a50a69873b9ffaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '459e4743ff3043f69408ca2095be3bd0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80219) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 22:24:00.679226 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6213d-46bb-46e3-a1e8-01320bf09164 service placement] REQUEST_METHOD: `GET` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 22:24:00.679512 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6213d-46bb-46e3-a1e8-01320bf09164 service placement] SCRIPT_NAME: `/identity` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 22:24:00.679793 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6213d-46bb-46e3-a1e8-01320bf09164 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80219) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 22:24:00.681587 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.fernet_utils [None req-91b6213d-46bb-46e3-a1e8-01320bf09164 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80219) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 22:24:00.693020 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6213d-46bb-46e3-a1e8-01320bf09164 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 22:24:00.694869 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6213d-46bb-46e3-a1e8-01320bf09164 service placement] RBAC: Authorization granted {{(pid=80219) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 22:24:00.704698 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: [pid: 80219|app: 0|req: 1433/2891] 10.222.0.44 () {60 vars in 1365 bytes} [Tue Jul 12 22:24:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 22:24:10.700488 dv-cinder-802955-7-iscsi devstack@keystone.service[80219]: DEBUG dbcounter [-] [80219] Writing DB stats keystone:SELECT=83 {{(pid=80219) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}